nm0765: er [0.5] so where we were last time [0.4] is we had er [0.8] written 
down a likelihood that we wanted to maximize [0.8] and [0.3] then we'd written 
down the first order condition so let's write remember the likelihood [1.5] the 
likelihood went like er [0.4] the Lagrangian sorry went like this [11.0] that 
was the [0.9] flow element of it [1.4] and er [2.0] there we had this extra 
stuff [13.5] that came from the [0.6] end conditions [0.8] now [2.3] we [0.3] 
realized that this Lagrangian didn't contain terms in the X-prime term [0.5] so 
this was a separable maximization [0.3] and all we needed to do [0.3] to 
maximize this lagra-, Lagrangian [0.6] was to differentiate it [0.2] at each 
time into T [0.2] and set those derivatives equal to zero [0.8] and if we did 
that [1.3] what we got from in here [0.9] was [0.5] D-F- [0.3] by-D-X [0.3] 
plus lambda-D-G- [0.9] by-D-X [0.3] is equal to [0.3] minus-lambda-prime [0.4] 
at all times little-T [1.5] okay [1.4] and that's just differentiating this 
expression [0.3] with respect to X [0.9] so that it's maximizing this term in 
the integral [0.3] at each time 
little-T [0.3] with respect to X [0.8] and the other condition we had [0.3] was 
D-F-by-D-U [0.5] plus lambda-D-G- [0.5] by-D-U [0.5] was equal to zero [0.6] at 
all times little-T [0.6] and that was again differentiating this point [0.4] 
thing in the integral here [0.4] with respect to U [0.2] at all times little-T 
[0.6] so we're maximizing this [0.2] with respect to U [1.6] now these two 
conditions have a special name they're called the [0.4] Hamiltonian conditions 
[4.9] and er [0.3] the reason they're called the Hamiltonian conditions a-, are 
apart from being named after a man called Hamilton [0.5] is that they er [0.9] 
sta-, are basically derivatives [0.2] of the function [0.5] F plus [0.4] lambda 
times G [0.6] with respect to X here [0.2] and with respect to U here [1.2] so 
we think of the function H which is F plus lambda times G [2.1] as [0.5] the 
thing that's being differentiated on the left-hand side [1.8] now [0.6] so 
these are like first order conditions if we wanted to maximize something [0.2] 
we'd take these first order conditions 
and solve them [0.7] but there's other things here [0.9] that we also need to 
worry about [1.0] the other things here that we need to worry about [0.7] come 
from [0.2] differentiating this Lagrangian [0.7] at the terminal time [1.1] you 
see [1.2] X at time T [0.2] appears here [0.4] then it appears here [0.7] so if 
we differentiate [0.2] the Lagrangian [1.0] with respect to X [0.2] at time T 
[0.5] we don't get an expression like this or an expression like this [0.6] we 
get [1.6] mu [0.5] minus [0.4] lambda at T [2.1] and that's [0.5] actually 
written right at the bottom of [0.2] of page two of your notes [0.9] now [0.3] 
and this has got to equal zero [1.7] now where did this [0.9] bit of the [1.0] 
Lagrangian come from [0.3] this bit of the Lagrangian [0.3] came from the 
terminal constraint [0.4] by our state variable [0.8] we had a ke-, terminal 
constraint which said that [0.3] X at time T [0.2] had to be greater than or 
equal to some number [0.3] X-nought [1.9] and this lambda at time T and this mu 
at time T [0.3] are two Lagrange multipliers for X at time T [0.8] mu is the 
Lagrange multiplier [0.2] that 
applies this constraint [0.5] lambda at T [0.4] is the costate variable [1.4] 
so this [0.4] condition here gives you a link between the two [7.6] so [0.2] i 
think probably the best thing to do now [0.5] is to use these conditions to 
solve a particular problem [0.8] so [1.2] let's [0.4] forget [1.3] this stuff 
[2.4] the stuff up there [0.6] and use the first order conditions [0.2] to 
solve a particular problem so you can see how they work in practice [1.0] so 
what i'm going to do is i'm going to take this stuff [0.2] and put it up there 
and then i'm going to solve the problem over here and you'll be able to see [0.
2] how they relate one to the other [0.5] so first of all i've got to be able 
to rub the blackboard off [1.8] excuse me [2.4] right [1.0] so let's get rid of 
that [1.8] and get rid of that [0.2] and we can get rid of that too [2.3] we'll 
leave the rest [0.2] up [1.8] now [3.2] the example i give you is on page three 
of the notes [0.5] and the example is one of a [0.9] a consumer trying to 
maximize their 
lifetime's utility [0.9] so what's the problem [0.6] the [0.2] the problem we 
face [0.8] is for a consumer who lives [0.5] from time period zero up to time 
period capital titl-, capital-T [0.8] whose [0.2] utility [1.1] is the [0.4] 
logarithm [0.6] of their consumption level [0.5] at time T [2.7] and er [1.8] 
they want to maximize this [2.8] by choosing a path of consumption [1.9] and 
their their ability to do this is constrained in certain ways [0.9] well they 
start off [0.2] their lives [0.4] with er a level of saving [1.4] say er ooh 
now they've got to have some positive saving [0.5] so let's call it [1.3] some 
level S-zero [1.1] and er [0.6] they also have a constraint on their [0.4] the 
way in which saving [1.7] changes through time [0.5] so [0.2] saving changes 
through time [0.3] that's [0.2] we look at the derivative of S at T [0.8] 
saving's 
going to go up [2.5] if we get [0.2] income from our current stock of saving so 
this is [0.3] the rate of interest on our current stock of saving [0.6] and 
saving's going to go down [1.2] if we eat some of our saving [0.8] right [1.9] 
so we're going to maximize our lifetime's utility [0.5] given we start off with 
this [0.3] fixed level of saving [0.5] given saving evolves in this way [0.3] 
and the final condition we have to have is we have to have [1.1] we die [0.6] 
not owning money [1.4] right [3.0] right [0.7] so this is a maximization 
problem [0.5] how are we going to apply these conditions [0.2] to this 
maximization problem [1.3] well what we want to do [0.2] is we want to think 
about what is a function F what is our function G here [1.9] so let's do that 
first of all [1.5] F [0.6] at time T [1.7] er it's an F let's just call it F [0.
9] is [0.8] a logarithm [0.7] of your consumptions [0.4] at time T [1.2] G [0.
4] 
in this particular case [1.1] it's the stuff [0.8] here [0.3] whatever that is 
[1.0] in this case it's [0.4] R of S-T [0.6] minus C of T [2.1] and that's it 
[2.6] so these [0.3] this is the thing you want to maximize [0.4] this is the 
thing [0.9] on the other side of the savings [0.5] equation [0.4] this is how 
savings change through time [1.1] G was the constraint on the rate of change in 
savings [1.0] clear [5.0] clear [0.3] 
sf0766: mm [0.2] 
nm0765: jolly good [1.0] so [2.2] er [0.8] how are you going to do this [0.5] 
well [0.3] let's do it in two stages first of all [0.3] let's write down a 
Hamiltonian for this [0.2] problem [0.5] so it's the logarithm [0.2] of C [1.5] 
plus [0.3] lambda [0.5] times R of S [0.2] minus C [2.9] that's [0.5] the er G 
bit [0.2] that's the F bit [2.2] now these conditions tell us how to solve the 
problem now [0.2] first 
of all [0.3] we have to do [0.6] D-H- [0.9] by- [0.5] D-X now at where X is the 
state variable [0.9] okay so let's pick someone namex [0.3] what's the state 
variable in this problem [1.4] what's the okay [0.2] what's the thing we can't 
choose [1.9] 
sf0767: consumption [1.3] [laughter] saving 
nm0765: really 
sf0767: and stuff 
nm0765: really consumption's the thing we're choosing for C for control 
sf0767: it's probably saving 
nm0765: okay S for state very good [0.6] so the thing we'd [0.3] we can't 
control the state variable is S [0.7] so [0.6] in this case the state variable 
we call saving [0.4] so you're going to have to do D-H-by-D-S [0.9] is equal to 
[0.7] minus- [0.2] lambda-prime [1.8] this one says [0.6] D-H [0.7] by the 
control variable which is consumption [1.5] is 
equal to zero [1.2] and so we need to take [0.5] these two equations [2.5] and 
now [0.3] i can rub that stuff off [2.7] and come over here [3.1] and [0.2] 
fill these in in more detail what is [0.4] H differentiated with respect to S 
[0.6] well [0.4] S isn't here but it's it's just this this is just the C [0.2] 
S only appears here in the H equation [0.6] so [0.2] D-H- [0.4] by-D-S [2.6] is 
actually [0.3] lambda times C R sorry [1.5] and that's got to equal minus-
lambda-prime [2.0] D-H-by-D-C [0.6] i know you like differentiating logarithms 
[0.4] so [0.2] you've got to differentiate this logarithm with respect to C [1.
5] one-over-C [0.8] and [0.3] there's a C term here so we can have [0.3] minus-
lambda [1.3] and that equals [0.7] 
zero [3.8] yes [1.1] thanks [0.6] so [2.3] we've got these two equations [0.6] 
that come from [0.5] the Hamiltonian condition but we also know one other thing 
[0.9] the other thing we know [0.4] is how saving evolves through time [0.4] so 
let's [0.4] not lose that [1.1] S-prime of T [0.4] is equal to R-S [0.2] minus 
C [3.7] we've got three equations [1.7] now i'm going to rub this off [6.4] and 
let's see what we've got [0.5] well the first thing we have [0.4] is that [0.3] 
we have a differential equation [2.3] for the costate variable [2.4] costate 
variable [0.5] is [0.4] comes straight from that [1.3] the second thing we have 
is we have a relationship [1.2] between a costate variable [1.1] and 
consumption [0.6] at time T [1.4] and the final thing we have is we have a 
relationship between saving [1.0] a differential equation for saving [1.0] and 
consumption [3.5] okay [1.9] so [0.2] once we've 
solved these [0.3] three [0.2] equations for the path of the costate variable 
[0.4] the path of consumption [0.3] and the path of saving [0.7] then we will 
have found [0.9] the optimal [0.8] path for consumption [0.4] in this problem 
[0.4] and the consequent optimal path for saving [1.4] okay [0.7] there are 
some constraints we haven't worried about yet [0.4] where saving starts up [0.
5] where saving finishes [0.9] we'll bing the bung those in later [1.5] clear 
[0.7] 
sf0768: i don't understand how the Hamiltonian relates to the Lagrangian [1.5] 
nm0765: [sigh] [1.4] okay let's go back [0.3] 
sf0768: sorry [0.5] 
sf0768: we had the Lagrangian [1.0] yes 
sf0768: mm 
nm0765: which consisted of [0.3] F [0.3] plus what [0.3] oh dear [0.2] hang on 
[0.2] lambda-G [1.9] plus some other stuff [1.0] 
sf0768: right 
nm0765: okay [1.1] and we said we needed to maximize that at each time period 
little-T [1.0] 
sf0768: oh so 
nm0765: so when we maximize the Lagrangian at each pime time period little-T [1.
3] we're basically maximizing the ham-, 
Hamiltonian [0.3] plus some other stuff [0.3] at each time period little-T [0.
7] 
sf0768: mm [0.7] 
nm0765: so that's how the Hamiltonian comes in [1.2] it's just basically [0.3] 
a way of writing [1.0] a part of the Lagrangian [1.5] 
sf0769: er okay [1.7] 
nm0765: is that all right 
sf0768: so it's not the differential of the Lagrangian then like 
nm0765: no [0.7] the derivative of the Hamiltonian [0.7] is the differential of 
the Lagrangian [1.9] 
sf0768: oh okay [0.7] 
nm0765: so we differe-, [0.2] when we differentiate the Lagrangian [1.1] we're 
doing the same thing we're differentiating the Hamiltonian [0.8] 
sf0768: so we just forget about all those other bits then [0.7] Lagrangian 
nm0765: basically [0.3] provided you know [0.6] the differential equation that 
gives you this the differential equation that gives you that [0.2] you can 
forget all that stuff about the [0.9] Lagrangian if you want if you just want 
to solve the problem [0.6] 
sf0768: okay [0.5] 
nm0765: so the only conditions you need to know [0.6] i've written on for you 
at the bottom of page two of the notes [0.9] provided you know that [0.4] you 
can solve optimal control problems [2.2] 
sf0768: okey-dokey [0.4] 
nm0765: okey-dokey [0.7] now any more questions while i'm at it [1.4] oh i've 
got a question for you [1.2] what's the solution for this differential equation 
[0.5] i'm going to choose the back row [2.7] ha ha ha [1.0] you haven't got the 
faintest idea 
sm0770: solution lambda [0.3] 
nm0765: ah nice try [0.4] suppose i gave you X-prime is equal to R-times-X [1.
3] it's a linear differential equation [0.3] with R [0.6] okay [2.8] a solution 
to this one is lambda at time T [0.4] is equal to an arbitrary constant times E-
to-the-power-R-T [1.7] if i differentiate this with respect to R i just get an 
extra R at the front ah minus-R [3.2] is that clear [2.6] [laugh] i'll take it 
slower [laughter] [2.1] i've got to rub off the problem 'cause we need all that 
stuff [5.2] so [2.1] we now have [0.9] these three equations [0.3] in these [0.
3] three unknowns [0.2] lambda-T we don't know [0.3] consumption we don't know 
[0.2] savings 
we don't know [1.3] we've got [0.8] differential equation here [0.4] in [0.2] 
lambda [0.9] it's a very simple differential equation it's a differential 
equation that appeared on right on the front of your notes on differential 
equations [0.5] it's a one variable [0.2] linear differential equation [0.5] if 
you just took those notes on differential equations and plugged everything in 
[0.5] you would find that the solution was [0.3] lambda at time T some 
arbitrary constant times E-to-the-minus-R-T [1.8] you can check that because if 
we do [0.2] differentiate lambda [0.5] at time T [0.4] you get minus-R-K [0.2] 
E-to-the-minus-R-T [1.7] right which is the same thing as minus-R times lambda-
T [2.7] yeah [1.8] brilliant [0.5] so the solution to this equation now i can 
write it down [0.3] is lambda at time T [0.6] is equal to an arbitrary constant 
times E-to-the-minus-R-T [2.3] so we know 
what lambda is [0.7] and because we know what lambda is we know what 
consumption is [2.3] let's get rid of that [1.7] consumption [2.1] is [0.3] one-
over- [0.2] lambda at time T [1.0] from this equation [2.3] and one-over-lambda 
at time T [0.7] is er [0.2] one-over-K [0.5] E-to-the-R-T [1.9] so what's 
happening to the optimal path of consumption [0.3] in this problem [0.6] well 
it's growing exponentially fast at the rate E-to-the-R-T [1.3] we've solved 
we've learned something [1.4] we don't know where [0.2] what this arbitrary 
constant is but we know the general pattern [0.5] of the path of consumption [1.
6] what about savings [1.3] well [0.9] the cha-, rate in change of savings [0.
6] is equal to R-S [0.5] minus consumption which is minus- [0.3] er -one-over-K 
[0.5] E-to-the-R-T [1.6] so now we have a differential equation [0.2] for 
savings at time T [1.7] which we need to solve [1.1] so [0.3] do i solve it [1.
9] i've just told you what the solution is [3.4] how are we going to do it [0.
3] well [0.8] we're 
going to do it in two stages we'll do it S-prime [0.3] minus R-S [0.4] is equal 
to minus-one-over-K [0.6] E-to-the-R-T [0.6] and then we multiply through by [0.
2] E-to-the-minus-R-T [0.8] so we get S-prime [0.3] E-to-the-minus-R-T [0.4] 
minus R E-to-the-minus-R-T S [0.7] is equal to [0.2] minus-one-over-K [2.8] and 
then we notice that this term here [0.2] is the derivative [0.8] of S [0.3] 
times E-to-the-minus-R-T [1.8] just applying the prod-, product rule [3.2] so 
[0.7] if i differentiate S with respect to T [0.2] and hold the E-to-the-minus-
R-T term constant [0.4] i get this bit [0.9] if i di-, [0.8] differenti-, hold 
the S constant [0.3] and differentiate [0.2] E-to-the-minus-R-T [0.3] with 
respect to T [0.3] i get minus-R [0.2] E-to-the-minus-R-T here [0.5] and then 
the S which i'm holding constant [0.7] right just from the product rule [2.3] 
do you believe me [1.0] 
sf0771: yeah [0.5] [laughter] [0.3] 
nm0765: do you un-, that's another que-, [0.3] do you understand [0.4] 
sf0771: yes [1.0] i think [0.6] 
nm0765: [laugh] [0.2] i can do it again 
sf0771: yeah just do the last two lines 
nm0765: okay from [0.2] this bit to this bit [0.3] 
sf0771: yeah [0.5] 
nm0765: okay [0.8] i said to you the 
problem differentiate this with respect to T [0.2] what are you going to do [0.
6] well first of all you'll differentiate S with respect to T [0.3] and hold 
this bit constant [0.6] if you did that [0.3] you'd get S-prime [0.6] E-to-the-
minus-R-T [1.3] then you'd hold the [0.8] S bit constant [0.5] and you'd 
differentiate this bit [0.2] so you'd get [0.2] minus-R [0.2] E-to-the-minus-R-
T [1.0] from differentiating this stuff [0.4] and then the S [0.2] that you're 
holding constant [2.1] brilliant [3.8] 
sf0772: yeah [0.4] 
nm0765: right [0.4] so [1.2] we know the differe-, differential of this with 
respect to T is [0.2] this [0.9] so to solve for S [0.9] oh er you're not going 
to like that i'd better come back over here [7.3] so for S [0.5] we say write 
[0.3] S [0.6] E-to-the-minus-R-T [0.4] is equal to [0.6] T-over-K [0.3] plus 
another arbitrary constant which we'll call C [0.8] or minus-T-over-K [1.3] 
just integrating both sides [0.4] we integrate this we get minus-one-over-K 
times T [0.2] plus another arbitrary constant [1.3] so finally [0.3] 
we know the path of saving [1.1] which is [1.2] E-to-the-minus-R-T [0.5] times 
T-over-K [0.6] with a minus there [1.4] plus [0.2] another arbitrary constant 
[0.2] er E-to-the-minus-R-T [3.4] no we don't [0.3] that's a minus-R-T [0.8] so 
these are pluses [2.3] okay [3.3] so [0.2] what have we done [2.0] we've [0.5] 
taken the differential equation [0.2] for lambda [1.0] and solved it [1.7] that 
allowed us to find what consumption was [0.8] and then we plugged in the [0.3] 
what we knew for consumption into the differential equation for saving [0.4] 
and solved that [1.1] so to sum up [0.3] we've got [0.4] lambda at time T [0.9] 
is [0.2] from here [0.8] an arbitrary constant [0.3] times E-to-the-minus-R-T 
[1.6] s-, consumption at time T [0.8] is the same arbitrary constant one-over-
it [0.7] times E-to-the-plus-R-T so consumption is growing [0.9] at the rate 
exponentially at the rate T [1.2] and we've got saving at [0.2] time T 
finally [0.3] from up here which is doing something funny [1.4] it's growing at 
the rate E-to-the-R-T [0.7] from this bit [0.6] then we have to knock off [1.2] 
K oh sorry knock off [0.4] T-over-K [0.7] times E-to-the-R-T here [0.2] so it's 
not clear what saving's doing [1.6] but if consumption's growing i guess 
saving's got to grow [0.6] so we've solved our problem [0.9] we took [0.7] the 
problem of finding the optimal path for capit-, [0.4] good consumption [1.5] 
wrote down the Hamiltonian conditions which gave us this equation and this 
equation [1.4] and we got er [0.6] this equation [0.5] from the constraint [0.
7] solved them [0.5] and we had to solve some differential equations [0.2] to 
do that [0.2] but it's not surprising you have to solve differential equations 
'cause differential equations [0.3] are equations which have [0.2] functions as 
solutions paths as solutions [0.4] and we're trying to find paths aren't we 
we're trying to find a path for consumption [1.6] and this is what we get [1.6] 
right [2.1] any questions [1.9] 
sf0768: 
yeah i lost the second when you went from that side of the board over here [0.
3] 
nm0765: this side [0.3] 
sf0768: yeah [0.5] 
nm0765: to this side [0.5] 
sf0768: mm [0.9] what happened there [0.2] 
nm0765: what happened there [0.3] i integrated both sides [0.5] 
sf0768: oh okay 
nm0765: so [0.3] if the derivative of this is a constant [1.1] if anything has 
a constant derivative [0.4] 
sf0768: mm [0.3] 
nm0765: then [0.3] the actual thing has got to be a linear function [1.1] 
sf0768: right [0.4] 
nm0765: so this is a [0.9] has a k-, [0.2] constant derivative [0.2] so when i 
integrate it up [0.3] it's got to have [0.2] K [0.8] it's got to be linear in T 
[2.2] 
sf0768: okay [0.2] 
sf0772: is that C consumption or a constant 
nm0765: if i [0.7] it's a constant [0.5] 
sf0768: mm okay 
nm0765: so if i do it backwards let's do it backwards [1.5] if i took that [1.
2] and then i differentiated both sides [0.6] with respect to T [4.1] what am i 
going to get if i differentiate this with respect to T [1.9] 
sf0768: 
nm0765: i'm going to get minus-one-over-K [1.8] which i've 
got here [0.2] minus-one-over-K [1.1] here i've got just that differentiated 
with respect to T [0.5] 
sf0768: mm-hmm [0.5] 
nm0765: so [1.2] you're happy that that's the same as that [0.6] it sorry this 
stuff [0.3] is the same as this stuff 
sf0768: mm-hmm [0.4] 
nm0765: so let's just rub out the [0.3] the brackets [5.1] oh i'd better write 
it out again [2.4] S E-to-the-minus-R-T [1.1] minus-one-over- [0.4] K [0.2] T 
[0.2] plus an arbitrary constant [0.3] [1.2] it's a big C [0.2] not a little C 
[0.4] in the notes it's [0.8] it's clear that it's a big C [3.8] right [2.7] 
but we haven't done anything [0.8] because [0.6] this solution that we've 
written up here [3.2] oh dear [6.3] right [0.4] this solution that we've 
written up here [3.0] has these arbitrary constants [1.3] and how and we got 
two arbitrary constants we got this K [0.7] and now we've got this [0.7] C as 
well this big C here [0.9] how are we going to 
solve them [1.1] well we know two things [1.8] we know [3.3] what the initial 
level of saving was it's some number [0.2] S-nought that's what you start your 
life with [0.9] you know something else as well [2.5] you know [0.6] that you 
can't end your life [1.3] with [0.9] zero saving so we know that [0.7] S at 
time capital-T [0.3] has got to be greater than or equal to nought [3.1] that 
gives us [0.7] a different [0.2] a condition [1.0] a la-, Lagrangian which says 
that lambda at time T [0.6] times S at time T minus zero [1.1] has got to be 
greater than or equal to zero [0.7] oh sorry [0.3] it's got to equal zero [2.8] 
we know that S this has got to be positive [0.4] we know that's got to be 
positive [1.2] so we can use [0.8] this condition [0.8] and this condition [0.
6] to tie down the arbitrary constants [2.9] basically 
what we're going to get [0.7] is going to get S at time zero is equal to S-
nought [0.6] S at time T [1.9] well [1.4] [2.5] right [1.9] so before we go on 
[0.6] any questions [2.4] yeah 
sm0773: where was that er [0.6] multiplier you put down on top of the [0.2] 
nm0765: this one 
sm0773: er that's right [0.7] 
nm0765: that came from the f-, last things i wrote down [0.6] right before i 
rubbed off [0.4] the [1.8] er Lagrangian [0.7] the Lagrangian [1.2] on page two 
of your notes [2.1] if you look right down at the bottom [1.0] you'll see what 
i wrote on the board [0.2] the very last line [0.5] mu is equal to lambda at 
time T [0.9] nu times ek-, ma-, times X at time T-minus-X-nought [0.3] oh 
that's a is equal to zero [1.2] so this [0.5] statement here [1.5] was the last 
thing i wrote down [0.8] before [0.5] is equivalent to [0.7] mu times [1.1] X 
at time T- [1.0] minus-X-nought [1.2] equals zero [0.2] it's equivalent to that 
thing [1.8] okay [3.0] we're going to deal with these terminal 
conditions [0.6] in greater depth now [0.4] so let's [0.5] move on 
nm0765: we've always talked about the constraint at the end [0.3] as being [0.
8] you can't die owe-, owning money [0.6] you can't die with negative savings 
[0.3] but we talked about [0.2] other sorts of terminal constraints that you 
might have [0.4] when you're doing these optimizations we talked about [0.2] 
steering a rocket [0.3] to the moon [0.6] and the terminal condition there was 
[0.3] you ended up at the moon [1.2] another sort of terminal condition you 
might have [0.4] is you [0.5] comes up when you er [0.2] fill the bath [0.6] 
when you fill the bath you have a [0.2] an absolute level that you want the 
bath to fill to [0.4] so [0.5] you know the bath is full [0.2] once the state 
variable has hit this particular level [1.7] final sort of terminal condition 
you might come across [0.3] would occur if you owned a machine [0.7] or a 
factory [0.3] or something like that [0.4] actually this isn't the final one 
there's another one [0.9] and the terminal condition there is [0.4] when do you 
quit [0.5] when do you decide [0.2] enough is enough and you 
want to leave the factory [0.6] so [1.3] that will be a terminal condition req-,
required you to decide [0.4] when to stop doing something [2.2] another ker-, 
terminal condition would arise [0.6] if [0.6] once you die [1.2] or once your 
machine is [2.5] dead [0.6] [laughter] [0.4] 
om0774: it's okay [0.3] 
nm0765: [laughter] i'm ruining your equipment [0.4] [laughter] once your 
machine is dead [1.0] er you get a certain value from it [0.7] so [0.8] er [0.
5] suppose we looked at an investment problem [0.7] and [1.2] you had to [0.4] 
plan your production for your firm so you had to choose the path of production 
[0.3] and then decide [0.3] when to scrap your machine [0.3] but once the 
machine and your investment and your plant were scrapped [0.2] you could take 
it and sell it to somebody [0.9] so [0.2] the amount which you could sell this 
stuff to at the end of the machine's life [0.5] would determine [0.8] when you 
wanted to stop [0.9] okay so [0.4] there are lots and lots of different sorts 
of ways in which the end point [0.3] might affect [0.5] what you wanted to do 
it 
with your problem [0.7] so let's look at [0.3] some of them now [0.9] er [0.5] 
so we're actually on page [1.2] four of your notes [2.3] so first of all [0.3] 
let's study the situation [0.3] where you have a finite [0.5] time [0.5] of [1.
0] shall we say death [1.8] capital-T [2.2] and what sorts of constraints might 
you face when you die [1.7] well [0.3] the easiest would be none [2.0] what 
that means is [0.2] that when you die [0.2] there is no constraint on the [0.6] 
your final position [0.3] your final value of the state variable [0.8] so none 
[1.4] I-E [0.7] X at time capital-T [0.3] is [0.3] unconstrained [7.3] if X at 
time capital-T is unconstrained [1.0] what would you expect the [0.4] Lagrange 
multiplier associated with X at time T capital-T to b-, take [1.3] well [0.6] 
we'd expect [2.6] a Lagrange multiplier was zero [0.3] i-, if [0.2] if 
something is unconstrained [0.5] it 
has a zero Lagrange multiplier [1.0] and that's exactly [2.2] the condition [1.
5] if you have a problem where you [0.8] don't care about w-, where you end up 
[0.5] then it must be the case [0.2] that your costate variable takes a zero 
value [0.2] at the end of your life [2.1] second sort of constraint you might 
have [1.0] you might have a a precise constraint [0.2] like the filling of the 
bath [0.6] which says [0.8] that once the bath is full it has to have a certain 
amount of water in it [0.2] and that's it there is no discussion [1.6] in that 
case [2.8] lambda at time capital-T is completely unrestricted [5.0] and we saw 
that sort of constraint coming in when we did [0.5] er [1.0] standard 
constraint optimization when we did standard constraint optimizations [0.5] 
equality constraints [0.5] meant lambda-T could take positive values negative 
values [0.6] we couldn't necessarily say that Lagrange multiplier will be 
positive or negative [0.5] when we had equality constraints [0.2] and the same 
comes through here [3.5] final condition [0.7] which we've seen before [2.9] X 
at time T has got to be greater than some number [0.6] so [0.2] that means you 
can't die owe-, owning money for example [1.9] and the constraint there [0.9] 
i've given you is that er [5.4] that looks wrong to me [4.3] no [1.6] lambda at 
time T [0.3] times [0.6] X at time T minus X-T [0.5] equals zero [1.2] which it 
[0.4] so that says either [1.5] the [0.4] costate variable [0.2] is zero [0.6] 
so this constraint isn't binding [0.3] and it's like X at time T is entirely 
free [0.9] or [0.9] X at time T [0.7] equals [1.1] X-subscript- [0.2] capital-T 
[0.5] so the constraint does bind [0.5] and then [0.5] lambda at time T [0.5] 
can be any value [1.2] so this is a [0.5] a way of writing this statement and 
this statement in some sort of composite 
form [2.9] okay [1.5] so that's w-, [0.7] these conditions [0.4] have a prop-, 
a name [0.5] they're called [0.5] transversality conditions [8.0] 
transversality conditions [0.2] are conditions that arise [0.2] from the [0.6] 
what goes on at the end of a life problem [0.3] end of the life of a problem 
condition [1.9] and they're very useful information [0.2] not in solving for 
the path usually [0.3] but they're useful information in solving for these 
arbitrary constants 
nm0765: right [2.3] now that's [0.7] er [1.1] transversality conditions [0.6] 
when we have a finite time horizon [1.4] but lots of economic problems [0.2] we 
don't want to have finite time horizons we want to allow consumers to live 
forever firms to live forever [0.3] we want to allow countries and planning 
problems to go on forever [1.0] and in problems like that we need a different 
set of transversality conditions so let's look at [0.8] er [1.1] 
infinite [1.4] horizon [2.8] problems now [2.2] infinite horizon problems we 
really want to say [0.2] the same things as these [0.4] but for the case where 
the capital-T is infinite [1.2] so [1.0] if there's no [0.6] constraint [0.6] 
what we want is then to [1.8] the limit [0.4] as T tends to infinity [0.4] of 
the costate variable of time capital-T [0.6] is zero [1.8] so instead of it 
being [0.3] zero at time t-, capital-T [0.3] we have it [0.6] going to zero [0.
3] as we go off to infinity [2.6] if we wanted a constraint which said [1.3] 
that in the limit [0.9] X at time T [0.4] equals some number [0.4] let's call 
it X-infinity [2.7] so [0.2] as T wen-, goes to infinity [0.3] the state 
variable [0.2] gets closer and closer [0.2] to this number X-infinity [1.0] 
then the equivalent transversality condition this one [0.6] over here [1.5] is 
just er [2.1] well let's see [0.5] 
lim- [0.6] T tends to infinity [1.6] lambda at time T [0.5] is unrestricted [5.
5] and the final one [0.4] again just rewriting that [0.5] do you think [0.4] 
for T tending to infinity [0.4] is the lim-T tends to infinity [0.5] X at time 
T [0.4] is greater than or equal to some number X [0.8] let's call it X-
subscript-infinity [1.3] what we require [0.5] is that the limit [0.7] as the 
capital-T tends to infinity [0.4] lambda-T [0.3] times X-T [0.5] minus [0.4] X-
infinity [0.8] equals zero [0.8] and all that's written in your notes [0.4] and 
[0.2] all this [0.2] is the same as all that just letting capital-T go off [0.
9] places [1.2] right [0.5] so we've dealt with [0.2] three sorts of end point 
restrictions that you might actually have in the your problems [0.5] and these 
three sorts of end point restrictions [0.3] just come from the Lagrange 
multiplier [0.9] nature of the problem [1.8] and we've done them in two cases 
we've done them where we in finite time horizons we've done them for infinite 
time horizons [1.4] but there are other sorts of end point constraints that 
we talked about [1.1] so let's look at one of them now [4.6] now [2.7] let's 
think of the machine problem [0.5] the machine problem [1.2] what you have is 
you have [0.6] a finite time horizon [1.3] with a pay-off which depends upon 
the time [0.3] the states and the control [1.6] and then [0.4] you get some pay-
off [0.6] from what happens at the end of your life [1.1] so we have er [0.9] a 
function [0.2] phi here [2.2] which depends upon [0.2] where you end up [0.4] 
so you can think of it as like being a prize [0.5] or [0.3] a reward [0.2] or 
something [0.9] so if you get big Xs here maybe this number's good [0.3] if you 
get small Xs here maybe this number's bad [0.4] but [0.7] it's what the stock 
of assets of [0.5] you own at the end of your life [0.2] are actually worth to 
you [0.2] in this problem [2.8] now if we wanted to maximize [0.7] this [0.2] 
object [1.6] we'd need to take account of the fact that [0.3] 
where we end up [0.5] has effects [0.4] on what [0.2] pay-off we're going to 
get from this problem [1.8] so [1.9] what [0.8] is the [0.4] relevant [0.8] 
terminal condition in this problem [0.2] well the relevant terminal condition 
[0.9] is that [1.1] lambda at time T [0.6] is equal to the derivative [0.6] of 
this function [0.9] evaluated at X at time T [1.5] so what does this say [1.3] 
well [0.7] suppose for some reason i was able to give you [0.4] one more unit 
of [0.3] final income [1.3] one more unit of it [1.9] how much more pay-off are 
you going to get [0.4] well [0.2] your pay-off is going to go up and find out 
[0.4] it's close to this derivative [1.6] so [0.2] this says [1.0] that lambda 
at time T [0.3] has got to be equal to the marginal [0.3] value to you [0.2] of 
extra income [0.3] at time T [1.9] okay [1.0] so this [2.4] yeah that's what it 
says [1.6] so [0.7] you can think about this lambda at time T [0.9] as being [0.
2] the value [1.6] of [0.9] extra [1.4] X [0.3] at [0.6] time [0.9] T 
nm0765: so we've been talking about these costate variables a lot [0.5] we've 
talked 
about [1.2] how we choose these costate variables [1.9] and [1.9] different 
constraints and how they affect the terminal value of these ker-, costate 
variables [1.4] but i haven't really told you yet [0.4] what these costate 
variables mean [0.4] what they actually [0.7] indicate [1.5] so let's [0.2] 
think about that [1.4] lambda-T [0.8] is the costate variable [8.8] what is it 
[5.3] well [1.8] we know that it's a Lagrange multiplier [7.1] and we know that 
Lagrange multipliers in some sense [0.2] measure the cost of a constraint [1.1] 
right [0.8] when we look at straight consumer optimization problems [0.3] the 
Lagrange multiplier there [0.3] represents the marginal utility of mon-, [0.2] 
money [0.8] the Lagrange multiplier [0.2] represents [0.2] how much it costs 
the consumer [0.2] to be constrained in its income if you could give the 
consumer more income [0.2] the constraint would be weaker [0.4] and the 
Lagrange multiplier measures that [1.7] this 
is a Lagrange multiplier for a different sort of constraint it's essentially [0.
2] the Lagrange multiplier for the constraint that says [0.4] X-prime of T [0.
4] is G-T- [0.3] X-U [2.2] right [0.4] this is the constraint that relates [0.
3] X [0.2] today [0.2] to values of X [0.2] tomorrow [3.8] so what does this [0.
5] Lagrange multiplier measure well it measures the cost of this constraint [0.
4] how much [0.4] having [0.2] X linked in this way to today and tomorrow [0.2] 
costs you [0.3] as an optimizer [1.8] now that doesn't seem particularly 
transparent [0.6] until you write it in a different way [1.0] lambda at time T 
[0.4] measures [3.3] the [1.4] cost or benefit [4.1] of [0.6] a small [2.0] 
increase [2.1] in [0.2] X at time T [2.5] on on [0.6] the [0.5] solution [2.3] 
to [0.4] a [1.7] an sorry [1.6] optimal [1.0] control [1.8] problem [3.4] so [0.
9] for some reason [0.2] so [0.3] what does lambda-T measure in an example [0.
8] think about er [2.1] think about the problem of sending your rocket to the 
moon [1.9] lambda at time T [0.2] essentially measures maybe the costs [0.4] 
associated with your fuel [0.2] at time T [1.2] so i s-, you're sending 
your your rocket to the moon [0.2] and somehow someone comes along to you [0.2] 
and says halfway to the moon [0.5] i can give you another gallon of petrol [0.
5] how much is that worth to you [1.8] well that's what lambda at time T 
measures [1.1] the m-, [0.3] the value to you somewhere along your problem [0.
2] of an extra unit of your state variable [0.2] an extra unit of petrol [0.3] 
an extra mile gone [2.7] clear [1.1] 
sf0768: what does lambda-T measure in your consumption problem 
nm0765: okay so it measures your money [0.5] because [0.2] what's the 
constraint in your consumption problem the constraint in your consumption 
problem [0.3] was S-prime is equal to R-S-minus-C [0.9] right [0.6] so i can 
going to give you [0.4] an extra unit's saving somewhere along the line [0.7] 
so it measures [0.5] the margin of utility of money to you [1.4] 
sf0768: mm-hmm [1.8] 
nm0765: yeah 
sf0768: mm-hmm [0.4] 
nm0765: but it doesn't measure the marginal utility of money at any old point 
of time [0.3] lambda at [0.4] T [0.2] repre-, represents the 
marginal utility of money to you [0.2] at time T [1.8] yeah [2.5] fine [2.4] er 
i can't think of a i can't think of a bath one [0.8] but it would measure 
something like [0.3] an extra egg cup full of water somewhere along the lay [1.
1] or er [2.7] yeah [0.2] er i think i've covered all the examples i understand 
[0.6] if you can think of another one [0.2] i'll have a go at it [1.3] [0.4] 
i've got another piece [1.5] okay [0.8] so we've done that [3.3] let's say more 
about these costate variables 
nm0765: consider a problem with discounting [1.6] now this is a some-, 
something that lots of mathematicians don't really [0.5] understand but er [0.
5] let let's er [4.5] we as economists do [0.8] right [1.6] this is a problem 
where [0.5] we have [0.6] a flow of utility [0.4] from consumption [0.2] at 
each point in time [1.0] but this [0.2] effect on our lifetime's income is 
discounted by the amount E-to-the-minus-R-T [0.5] so one unit of consumption 
you [0.2] to you at time zero [0.7] is worth a lot le-, more [0.4] to you than 
one unit of consumption at time [0.2] 
capital-T [0.4] because [0.7] you're impatient [0.4] and so this E-to-the-minus-
R-T [0.3] makes [0.2] future consumption worth less to you [0.2] than current 
consumption [0.4] right [2.1] fine [1.3] so [0.4] lambda at time T [0.7] we've 
already talked about thanks to er [0.3] namex [0.8] measures the [1.1] 
additional value to you [0.3] of an extra unit of income [0.3] at time T [0.6] 
right [1.9] so this is [1.4] value [1.5] to you [2.6] of [0.8] extra [1.1] 
income [2.0] at [1.3] time [0.7] little-T [2.7] but [2.2] that's not actually 
what you're interested in [1.1] because [1.7] you're now in period zero [0.3] 
trying to decide [0.3] what you're going to do for the whole of your life [0.5] 
and lambda at time T measures [1.1] how you feel about an extra unit of income 
[0.5] at time [0.8] little-T [0.4] given you're currently at time zero [3.0] so 
[0.9] you might ask me the question [0.7] you might [1.1] if i gave [0.2] how 
about my feeling about this extra unit of income at time [0.2] T [0.2] rather 
than at time little-zero [1.7] well we define this thing called M at [0.3] T [0.
5] to be [1.8] E-to-the-power-R-T times lambda-T [2.1] and what does that do [0.
6] well it scales up [1.4] the the value of 
to you of income at time T [0.3] by the discount factor [0.2] E-to-the-R-T [1.
0] so it [1.1] undoes the effect of the discounting [1.8] and [0.9] says to you 
[1.8] oh let's let's [0.4] let's step back [1.5] what it does [0.8] is says to 
you [0.3] okay [0.8] an extra unit of income to you [0.3] at time T [0.7] is 
worth this much to you at time zero [1.0] to work out how much it's worth to 
you at time T [0.4] we have to scale it up again [0.8] 'cause when you get to 
time T [0.4] the [0.7] time T now is the present and so you have to undo the 
effect of the discounting [1.4] so this thing here is called the [0.2] the 
current [1.7] marginal [2.3] valuation [6.4] now this thing here is going to be 
jolly useful for us in solving economic [0.7] optimal control problems [0.5] 
because [1.2] in economic optimal control problems [1.3] we often find that we 
have discounting [1.4] so it's often [1.3] okay [0.7] it's often [1.2] useful 
[2.1] to [0.5] eliminate [2.3] the costate variable [1.7] lambda at time T [1.
0] and [1.0] replace it with [3.8] M at time T [0.5] the current [1.7] marginal 
[3.1] valuation [3.2] so it's time for an example of that [4.5] and we'll do 
that next time