The Lorenz Attractor

Or Chaos for beginners

Marcus McGuff, A.Mus.D.
Austin Community College

Introduction

Edward Lorenz (1917 - 2008) was an American mathematician and meteorologist who published a paper in 1962 that was one of the pioneering efforts into what has become known as "chaos theory".  In it, he wanted to consider the feasibility of long-term weather forecasting by examining a mathematical model of a simplified physical system.  This led to a system of first order autonomous differential equations whose solutions turn out to have some very odd behaviors.

Since modeling the actual weather is insanely difficult, scientists have studied several simplified models, one of which is the "convection cell".  This is a long, thin cell of negligible width (eliminating one spatial dimension for simplicity) of some height that is heated on the bottom (as the earth heats the air above it) and cooled at the top (as the atmosphere radiates heat into space).  This can be considered a very simplified model of the atmosphere.  If the air could not move, then heat would simply radiate upwards via conduction.  However, when air is heated it expands, becoming less dense.  As all balloonists know, lighter/hotter air rises.  If the heating isn't too intense, the heat will leak out of the pocket of air (via conduction) and it will lose buoyancy before it can force its way upwards, so there isn't much movement.  However, if the heating is more intense, the pocket of air will retain enough heat/buoyancy to force its way upwards before it loses steam (well, heat anyway).  Once it gets high enough, there will have been enough heat loss for the pocket to be colder and denser and it will tend to fall again.  So, as air cools off higher in the atmosphere, it gets denser and tends to fall.  This motion of the hotter air rising from the ground while the colder air sinks out of the higher atmosphere is called convection.  You can see the same sort of phenomena when you heat a container of water; at first, the heat will simply be conducted upwards through the water, but as you continue to add heat, the water will began to circulate and begin roiling.  This convection causes air currents that lead to much of the weather patterns we see.  The convection cell is a simplified model of this system.

In his paper, Lorenz derived a system of 3 first order autonomous equations from earlier much more complicated equations developed to study fluid flow and thermal energy, making several changes of variable and approximations along the way.  The derivation is beyond the scope of this course, so we'll skip straight to his set of equations (you can thank me later):

	X' (t) = σ (-X(t) + Y(t))
Y' (t) = r X(t) - Y(t) - X(t) Z(t)
Z' (t) = -b Z(t) + X(t) Y (t)
To understand the meaning of the variables here,  take a look at the following simulated convection cell:

           
 
      t≡ time (actually, time with some rescaled units)

X(t)≡ This gives the direction and speed of the convection "roll", where a positive value means the air is circulating clockwise (illustrated by the circulating arrows).

Y(t)≡ This gives the temperature difference between the left and right sides of the box, where a positive value means the left side is hotter (illustrated by the color gradient across the box).

Z(t)≡ This gives the deviation of the temperature function from a straight, linear (conductive) increase (illustrated by the color gradient vertically in the box).  Z is always positive, but the larger Z is, the more quickly the temperature will change at the boundaries.

Notice, there are also 3 constants:

σ≡ This is the Prandtl number, which is the ratio of momentum diffusivity (kinematic viscosity) and thermal diffusivity; in other words, it is a measure of the relative rates at which particles can move to the rate at which heat is transferred.  (When σ is small, it means that the heat diffuses very quickly compared to the velocity/momentum, so the pockets of air lose heat more quickly than they can move upwards, for example.)

r≡ This is the Rayleigh number, which is a dimensionless number associated with buoyancy driven flow (also known as free convection or natural convection).  When the Rayleigh number is below some critical value for a fluid, heat transfer is primarily in the form of conduction; when it exceeds that value, heat transfer is primarily in the form of convection.  This is the most important one for our model (physically and mathematically).

b≡ This is a constant related to the pressure of the fluid/gas.

Analyzing the system

The overall behavior of solutions

Notice that the Lorenz equations are autonomous (good news) and nonlinear (not so good).  Recall that one way to study this kind of system is to find the equilibrium points and the and examine the eigenvalues of the local linearization found by the Jacobian matrix at each of these points.

So, first, find the equilibrium points for this system in terms of σ, b, and r.

How many are there?  Okay, if you read any further, you'll know the answer to this, but notice that the answer depends on the value of the constants.  What restrictions do we need to place on our constants if we want more than one critical point (which is exactly what we want; only one would be boring)?  If there are more than 1, how are the extras related to each other?

For discussion purposes, we will examine the same set of values for the constants that Lorenz used in his paper:  σ=10, b=(8/3), r=28.  We will also take (5,5,5) as our initial condition point.  This gives 3 equilibrium points.

Use the Lorenz attractor in 3 dimensions tab of the widget at the end of this lab to look at the solution curve for these values; if you hover over each of the equilibrium points (in blue), you will see a list of its eigenvalues there (well, of the local linearization).  For each equilibrium point, make a list of its eigenvalues and make some initial conclusions about what you think the type of eigenvalues tells you about the behavior of the graph at that point.    Since all the examples we have looked at in class have been in only 2 dimensions, you will need to do a little creative/educated guessing.

In working with 3-dimensional systems, it is sometimes useful to look at the projections of the solution into the 3 coordinate planes.  Using the same settings as above, these look like:

You can work with these more directly in the widget at the end; take a look at the first part on "Projections into coordinate planes".  Compare these graphs to what you see in the 3-dimensional graph until you feel you have a reasonably good feel for the overall shape (if not many details).  Be sure you notice where the equilibrium points are in these graphs (they are in blue on the widget).

Do you think the graph intersects itself?  If so, does it occur often or only occasionally?  Do you think the graph ever intersects the equilibrium points if you wait long enough?

A more careful look at things

While this does make for a pretty cool looking picture, we need to look at this in a bit more detail.  Take a look at the time plots for X, Y, and Z to understand what is going on better:

On a relatively short time scale:
        

Or, on a longer time scale:
             
These are fairly complicated graphs, but several questions might be asked:

These graphs look vaguely like sound waves (complicated ones), so are they periodic?  For this to occur, the graph would have to intersect itself.

X(t) and Y(t) look awfully similar; are they the same or different?  If different, are they "close enough" to behave in basically the same way?

How exactly (or even approximately) do these graphs relate to the graph in 3-dimensions of the solution curve?

Notice that X(t) and Y(t) stay negative for a bunch of oscillations, then they suddenly "flip" and become positive, where they stay for a little bit, then briefly flip down, then back up to positive for a bit more, etc.  What happens in the 3-dimensional (or the projections) when this occurs?

So, what does this behavior have to do with the physical system we are modeling?  What physical changes is this a reflection of?  What physical support do you have to add to your answer to question 2?  What does your answer to questions 4 and 5 tell you about the behavior of  mini-weather system?

What's going on with all these swirly things anyway?  Are these really circles or not?

It turns out that some of these questions are very difficult to find and prove the answer to mathematically.  However, combing a bit of physical intuition with the math can definitely help you with the others.  The full solution curve for the two time periods shown above is given by:

t=8
t=50 
(You can drag these graphs around with your mouse to look from different angles; if you Control-Drag with your mouse, you can zoom in/out and Shift-Drag moves it around.)

Of course, a better way to examine this is to head to the widget at the end of the lab and experiment away.  It gives you several different views, with some hints to help you see what's going on.  For example, notice the coloring of the curves; the graphs are colored according to the time, so you can see what happens at the same time by matching colors (though they may repeat, so be careful you don't think all the yellow parts have the same time; this is more to help you trace the path).  Also, follow the moving red dot; it always shows "now" on all graphs (which you can use the slider to change).  The arrows also give you some hints as well.

Also, be sure to check out the "Lorenz Simulation" tab.  This is where I demonstrate my wonderful animation skills to animate our simulated weather system.  The color gradient across the background simulates what is going on with Yand Z, while the whirling circles simulate X.  I suspect you will find it easier to watch what's going on with X; you can view either the time domain graphs or the projections of the solution onto the coordinate planes while the simulation is running (along with the omnipresent moving red dot to help you track when you are).  Notice that the 3 circles are not in sync; this is because they have slightly different initial conditions. The middle one uses the initial conditions you supply, while the left one is "perturbed" by a small amount you set in a random direction and the right one is perturbed by twice as much.  This is meant to give you some idea how nearby air masses might interact with each other, according to these simplified equations.  (We will examine the effects of initial conditions on the solution in more detail in the next section.)  One word of warning about the simulation:  It really only accurately reflects the signs of the three functions; this gives a good qualitative idea about what's happening (well, maybe not so much for Z), but it isn't meant to be accurate beyond that.

Write a brief paragraph or two describing what you observe, paying attention to the questions I asked above.  I am particularly interested in the relationship you see between the time domain solution curves and the "phase plot" of the curve in 3-dimensions and how these relate to the physical system.

So, what about the chaos?  Well?

So, I'm sure that the real reason many of you are reading this is that I mentioned "chaos" in the title.  Everyone has heard about it and it sounds really cool, so what about it?  Well, first, let's be clear about what chaos does and does not mean.  In the mathematical sense, it does not mean randomness.  Chaotic systems are deterministic in nature.  This means that they are completely specified by their initial conditions; if you know where a system started from, you can simply follow the equations to find out its future behavior (just like all the other differential equations we have studied in this course).  There is nothing magical or truly random about it.  However, the property that chaotic systems do have that is different than non-chaotic ones is that they are fundamentally sensitive to very minute changes in their initial conditions.  So, a slight change in the initial temperature of the system, for example, might lead to wildly different long-term behavior.  It isn't simply that the two solutions diverge from each other over time (not that uncommon, really), but the initial close relationship between them breaks down until there is no discernible relationship between them anymore, i.e., it appears that the differences between them are just random.  Later, they may be far apart or close together but headed in different directions or even converge for a little while.  In a "perfect" (but fairly boring) world, there is no randomness here; once you know the exact initial conditions of a system, you can confidently predict its future behavior exactly (assuming your model is good enough).  Of course, the catch here is that any minor difference in the initial conditions may lead to significantly different behavior down the road.  Now, if this were simply a matter of the two solutions drawing further apart over time, we could make adjustments to allow for "solution drift".  The interesting thing about chaotic systems is that they can coincide exactly for a significant time, then suddenly break apart and re-intersect many times later, appearing totally independent of each other.  

The situation is even worse than it seems for chaotic systems.  Because most of these systems of equations cannot be solved exactly, we have to use numeric methods to compute the solutions.  A little bit of thought should show you why this is a problem:  When you solve a system or equation numerically, you have to make approximations.  Even extremely accurate calculations will be off slightly, which might be enough to shift over to a different solution curve for the next step in the calculation; because of the chaotic behavior, this means that your final answer may not be even mildly related to the solution you are computing.  Those of you paying attention should be thinking about the fact that my computer widget (or anybody's cool pictures of the Lorenz curve) uses just such a numerical method to draw these.  Well, yeah, you caught me.  On the other hand, even if I were obsessive about making things as accurate as the best super-computer can be, my solution would still not be truly accurate.  That is, of course, the whole point.  So, what you are looking at here is a lie; fortunately, it's a fairly accurate lie, since it shows the basic qualitative behavior of the system.  There really is a pattern here.

Just to give you an idea of what all this is about, below is a graph of X(t) for 3 different initial conditions (same setup as before):  



Notice that you can only see one plot, which is what you would expect for solutions that started so close together.  Let's look at things a bit later:



Notice that the curves are starting to diverge around 9.5 or 10 and they really start looking different by 12.  How about later?





I hope you see the point by now.  Well before we get to 50, things are looking pretty random.  The 3-D plot at t=24 would be:



There is an interesting theorem (the Poincaré–Bendixson Theorem) that says that any orbit  in a compact region of the state space of a 2-dimensional planar continuous dynamical system approaches either a fixed point or a periodic orbit.  The Lorenz equations (the graph is known as the "Lorenz attractor" or the "strange attractor") is neither of these; its trajectory is infinite (so it can't be periodic), but it never converges to the critical points.  The practical result of this is that systems of continuous differential equations can only lead to chaotic behavior in 3 dimensions or more.

Go to the "Perturbations" part of the widget and try different perturbations from the same initial point used above (don't get too crazy).  What do you observe about the length of time it takes the graphs to significantly diverge?

Before you get the wrong impression, the Lorenz equations are not chaotic for all values of the constants.  True chaotic behavior seems to set in around r=28 (keep the other constants the same).

Compare the behavior of several solution curves for each r values of 2, 12, 24, and 48.  Which of these seems non-chaotic (give some justification) and what happens to the ones that are not chaotic?  In the non-chaotic situations, explain what the physical significance of that might be in the model.

Conclusion

Before you totally give up on ever being able to predict the weather (assuming it is, in fact, a chaotic system), be sure you understand what chaos theory does and does not claim.  Notice that it does not say that future behavior of a solution in totally unpredictable.  In fact, if you look at our graphs, you see that small perturbations in initial conditions don't actually seem to matter much for a certain period of time; the graphs basically overlap.  It's long-term when these graphs truly seem unrelated.  This means that short-term predictions are actually quite feasible (and short term weather forecasts are even right much of the time).  However, this does not hold out much hope for long-term prediction of the behavior of chaotic systems.  In those cases, random guessing (well, educated guessing at least) or even bad approximations might be just as accurate.

Problems and tools

Problems

First, don't forget to answer the questions in the text of the lab.

For the set of constant values given, investigate the dependence on the Lorenz curve and time solutions on broadly different initial conditions.  Does it seem to make much difference in the long run where you start from?

Keeping r and b the same as above, see what you can discover about the effect of σ on the Lorenz curve.  (Keep the same set of initial conditions so you aren't varying everything at once.  If you get an error when doing this, don't freak out.  There is some fairly heavy-duty work going on behind the scenes while still trying to keep the animation usable, so some choice for parameters just require more computation.)

Fool around with the parameters and show me a graph (with parameter and initial conditions settings) you think is particularly interesting or artistic.  Go wild...

Tools - The Lorenz Attractor Widget