Entropy (arrow of time)

Encyclopedia : E : EN : ENT : Entropy (arrow of time)

Unsolved problems in physics: Arrow of time: Why did the universe have such low entropy in the past, resulting in the distinction between past and future and the second law of thermodynamics?

Entropy is the only quantity in the physical sciences that "picks" a particular direction for time, sometimes called an arrow of time. As we go "forward" in time, the Second Law of Thermodynamics tells us that the entropy of an isolated system can only increase or remain the same; it cannot decrease. Hence, from one perspective, entropy measurement is thought of as a kind of clock.

By contrast, all physical processes occurring at the microscopic level, such as mechanics, do not pick out an arrow of time. Going forward in time, we might see an atom moving to the left, whereas going backward in time, we would see the same atom moving to the right; the behavior of the atom is not qualitatively different in either case. In contrast, we would be shocked if a gas that originally filled a container evenly, spontaneously shrinks to occupy only half the container.

Contents

1 Overview
2 Example
3 Maxwell's demon
4 Correlations
5 Current research and cosmology
6 See also

Overview

The reader may have noticed that the Second Law allows for the entropy remaining the same. If the entropy is constant in either direction of time, there would be no preferred direction. However, the entropy can only be a constant if the system is in the highest possible state of disorder, such as a gas that always was, and always will be, uniformly spread out in its container. The existence of a thermodynamic arrow of time implies that the system is highly ordered in one time direction, which would by definition be the "past".

Unlike most other laws of physics, the Second Law of Thermodynamics is statistical in nature, and its reliability arises from the huge number of particles present in macroscopic systems. It is not impossible, in principle, for all 10²³ atoms in a gas to spontaneously migrate to one half of container; it is only fantastically unlikely -- so unlikely that no macroscopic violation of the Second Law has ever been observed.

The thermodynamic arrow is often linked to the cosmological arrow, as according to the Big Bang theory, the Universe was initially very hot with energy distributed uniformly. As the Universe grows its temperature drops, which leaves less energy available to perform useful work in the future than was available in the past. Thus the Universe itself has a well-defined thermodynamic arrow of time. But this doesn't address the question of why the initial state of the universe was that of low entropy. If cosmic expansion were to halt and reverse due to gravity, the temperature of the Universe would once again increase, but it's expected that entropy would continue to increase.

Example

As an example, let's say you filmed a ball near the Earth as it moved up, slowed gradually to a stop, and then fell back down to the same position it started in. If all of the laws of physics were symmetrical you would expect it to take the same amount of time for the ball to go up as it did for it to come down, but as it turns out your film would show it taking longer on one leg of the journey than it did on the other. If we were to look at just this stretch of film you might conclude, incorrectly, that Gravity was not symmetric. However, if your film were good enough to capture the motions of each air molecule it would tell a different story.

In that case what you would see would be that, as you play the film in one direction, the ball would be constantly colliding with the molecules of air in front of it, transferring some its momentum to them, and those collisions would be slowing it down. If you were to play the film in the reverse direction it would show molecules of air striking the ball from behind, and speeding it up. Once you take into account both gravity and momentum the film shows that they are both symmetrical forces. This still leaves the asymmetry in the film unaccounted for. The second law of thermodynamics explains that asymmetry.

The law of thermodynamics relates to the entropy of a system (in this case the 'system' is the ball, the surrounding air, and the Earth's gravity). It states that closed systems will tend to change from a state of higher order to a state of lower order. In the above example the system starts out with a large amount of Kinetic energy (speed or motion) concentrated in the ball. As it moves through the air the ball transfers some of that energy to the air in the form of heat. Eventually the ball ends up in the same place it started out in, but the ball has less energy than it started out with and the air has the same amount more. A more orderly state has become less orderly since the energy has become less concentrated and more diffused.

To take the example one step further, let us say you continued to film long enough to see the ball strike the ground. In that case you would see an end state where all of the kinetic energy has been turned into heat and sound waves and transferred from the ball into the ground and air. There are many other examples of such processes, including friction and heat dissipation. Any irreversible process can be used as an "arrow" that points to a direction in time.

Maxwell's demon

In 1867, James Clerk Maxwell introduced a now-famous thought experiment that highlighted the contrast between the statistical nature of entropy and the deterministic nature of the underlying physical processes. This experiment, known as Maxwell's demon, consists of a hypothetical "demon" that guards a trapdoor between two containers filled with gases at equal temperatures. By allowing fast molecules through the trapdoor in only one direction and only slow molecules in the other direction, the demon raises the temperature of one gas and lowers the temperature of the other, apparently violating the Second Law.

Maxwell's thought experiment was only resolved in the 20th century by Leó Szilárd, Charles H. Bennett, and others. The key idea is that the demon itself necessarily possesses a non-negligible amount of entropy that increases even as the gases lose entropy, so that the entropy of the system as a whole increases. This is because the demon has to contain many internal "parts" if it is to perform its job reliably, and therefore has to be considered a macroscopic system with non-vanishing entropy. An equivalent way of saying this is that the information possessed by the demon on which atoms are considered "fast" or "slow", can be considered a form of entropy known as information entropy.

Correlations

An important difference between the past and the future, which has often been overlooked in past discussions, is that in any system (such as a gas of particles) its initial conditions are usually such that its different parts are uncorrelated, but as the system evolves and its different parts interact with each other, they become correlated. For example, whenever dealing with a gas of particles, it is always assumed that its initial conditions are such that there is no correlation between the states of different particles (i.e. the speeds and locations of the different particles are completely random, up to the need to conform with the macrostate of the system). This is closely related to the Second Law of Thermodynamics.

Take for example (experiment A) a closed box which is, at the beginning, half-filled with ideal gas. As time passes, the gas obviously expands to fill the whole box, so that the final state will be a box full of gas. This is an irreversible process, since if the box is full at the beginning (experiment B), it will not become only half-full later, except for the most unlikely situation where the gas particles have very special locations and speeds. But this is precisely because we always assume that the initial conditions are such that the particles have random locations and speeds. This is not correct for the final conditions of the system, because the particles have interacted between themselves, so that their locations and speeds have become dependent on each other, i.e. correlated. This can be understood if we look at experiment A backwards in time, which we'll call experiment C: now we begin with a box full of gas, but the particles do not have random locations and speeds; rather, their locations and speeds are so particular, that after some time they all move to one half of the box, which is the final state of the system (this is the initial state of experiment A, because now we're looking at the same experiment backwards!). The interactions between particles now do not create correlations between the particles, but in fact turn them into (at least seemingly) random, "canceling" the pre-existing correlations. The only difference between experiment C (which defies the Second Law of Thermodynamics) and experiment B (which obeys the Second Law of Thermodynamics) is that in the former the particles are uncorrelated at the end, while in the latter the particles are uncorrelated at the beginning.

In fact, if all the microscopic physical processes are reversible (see discussion below),then the Second Law of Thermodynamics can be proven for any isolated system of particles with initial conditions in which the particles states are uncorrelated. In order to do this one must acknowledge the difference between the measured entropy of a system - which is dependent only on its macrostate (its volume, temperature etc.) - and its information entropy, which is the amount of information (number of computer bits) needed to describe the exact microstate of the system. The measured entropy is independent of correlations between particles in the system, because they do not affect its macrostate, but the information entropy does depend on them, because correlations lower the randomness of the system and thus lowers the amount of information needed to describe it. Therefore, in the absence of such correlations the two entropies are identical, but otherwise the information entropy will be smaller than the measured entropy, and the difference can be used as a measure of the amount of correlations.

Now, time-reversal of all microscopic processes implies that the amount of information needed to describe the exact microstate of an isolated system (its information entropy) is constant in time. If there are no correlations between the particles initially, then this is just the initial thermodynamic entropy of the system. However, if these are indeed the initial conditions (and this is a crucial assumption), then such correlations will form with time, and for a time which is not too long - the correlations between particles will only increase with time; therefore, the measured entropy must also increase with time [link]. (Note that "not too long" in this context is relative to the time needed, in a classical version of the system, for it to pass through all its possible microstates - a time which can be roughly estimated as [\tau e^S], where [\tau] is the time between particle collisions and S is the system's entropy. In any practical case this time is huge compared to everything else)

Current research and cosmology

All phenomena that behave differently in one time direction can ultimately be linked to the Second Law of Thermodynamics. This includes the fact that ice cubes melt in hot coffee rather than assembling themselves out of the coffee, that a block sliding on a rough surface slows down rather than speeding up, and that we can remember the past rather than the future. This last phenomenon, called the "psychological arrow of time", has deep connections with Maxwell's demon and the physics of information; In fact, it is easy to understand its link to the Second Law of Thermodynamics if we view memory as correlation between brain cells (or computer bits) and the outer world. Since the Second Law of Thermodynamics is equivalent to the growth with time of such correlations, then it states that memory will be created as we move towards the future (rather than towards the past).

Despite the statement above, some processes which involve high energy particles and are governed by the weak force (such as $K-meson$ decay) defy the symmetry between time directions. However, all known physical processes do preserve a more complicated symmetry (CPT symmetry), and are therefore unrelated to the Second Law of Thermodynamics, or to our day-to-day experience of the arrow of time. A notable exception is the wave function collapse in quantum mechanics, which is an irreversible process. It has been conjectured that the collapse of the wave function may be the reason for the Second Law of Thermodynamics.

It currently seems that the ultimate reason for a preferred time direction is that the universe as a whole was in a highly ordered state at its very early stages, shortly after the big bang, and that any fluctuations in it were uncorrelated. The question of why this highly ordered state existed, and how to describe it, remains an area of research. Currently, the most promising direction is the theory of cosmic inflation.

According to this theory our universe (or, rather, its accessible part, a radius of 13 billion light years around our location) evolved from a tiny, totally uniform volume (a portion of a much bigger universe), which expanded greatly; hence it was highly ordered. Fluctuations were then created by quantum processes related to its expansion, in a manner which is supposed to be such that these fluctuations are uncorrelated for any practical use. This is supposed to give the desired initial conditions needed for the Second Law of Thermodynamics.

Our universe is probably an open universe, so that its expansion will never terminate, but it is an interesting thought experiment to imagine what would have happened had our universe been closed. In such a case, its expansion will stop at a certain time in the distant future, and it will then begin to shrink. Moreover, a closed universe is finite. It is unclear what will happen to the Second Law of Thermodynamics in such a case. One could imagine at least three different scenarios (in fact, only the third one is probable, since the first two require very simple cosmic evolution):

It could be claimed that in such a case, the quantum fluctuations - which in the meantime have involved into galaxies and stars - will be in superposition in such a way that the whole process described above is reversed - i.e. the fluctuations are erased by destructive interference and total uniformity is achieved once again. Thus the universe ends in a big crunch which is very similar to its beginning in the big bang. Because the two are totally symmetric, and the final state is very highly ordered - entropy has to decrease close to the end of the universe, so that the Second Law of Thermodynamics is reversed when the universe shrinks. This can be understood as follows: in the very early universe, interactions between fluctuations created entanglement (quantum correlations) between particles spread all over the universe; during the expansion, these particles became so distant that these correlations became negligible (see quantum decoherence). at the time the expansion halts and the universe starts to shrink, such correlated particles arrive once again at contact (after circling around the universe), and the entropy starts to decrease - because highly correlated initial conditions may lead to a decrease in entropy (one may view this as the following: as distant particles arrive, more and more order is revealed because these particles are highly correlated with particles which have arrived earlier).

It could be that this is the crucial point where the wavefunction collapse is important: if the collapse is real, then the quantum fluctuations will not be in superposition any longer, but rather they had collapsed to a particular state (a particular arrangement of galaxies and stars), thus creating a big crunch which is very different from the big bang. Such a scenario may be viewed as adding boundary conditions (say, at the distant future) which dictate the wavefunction collapse[link].

Finally, highly smooth initial conditions may lead to a highly non-smooth final state. Highly non-smooth gravitational systems tend to collapse to black holes, so the wavefunction of the whole universe evolves from a superposition of small fluctuations to a superposition of states with many black holes in each. It may even be that it is impossible for the universe to have both a smooth beginning and a smooth ending. Note that in this scenario the energy density of the universe in the final stages of its shrinkage is much larger than in the corresponding initial stages of its expansion (there is no destructive interference, unlike in the first scenario described above), and consists of mostly black holes rather than free particles.

In the first scenario, the cosmological arrow of time is the reason for both the thermodynamic arrow of time and the quantum arrow of time. Both will slowly disappear as the universe will come to a halt, and will later be reversed.