ARTICLE

pubs.acs.org/jchemeduc

Entropy: Order or Information

Arieh Ben-Naim*
Department of Physical Chemistry, The Hebrew University of Jerusalem, Edward J. Safra Campus, Givat Ram, 91904, Jerusalem, Israel

ABSTRACT: Changes in entropy can sometimes be interpreted in terms

of changes in disorder. On the other hand, changes in entropy can always
be interpreted in terms of changes in Shannons measure of information.
Mixing and demixing processes are used to highlight the pitfalls in the
association of entropy with disorder.
KEYWORDS: First-Year Undergraduate/General, Upper-Division Un-
dergraduate, Physical Chemistry, Misconceptions/Discrepant Events,
Thermodynamics

I n 1865, Rudolf Clausius introduced the concept of entropy in

connection with his work on the mechanical theory of heat.
The original denition of entropy in terms of heat transferred and
association of entropy changes with changes in order is, in
general, invalid. Instead, the same processes can be consistently
interpreted in terms of changes in the amount of missing
absolute temperature did not oer and, in principle, could not information.6
oer any molecular interpretation of the entropy. Ludwig Boltz- Consider the following chain of reasoning that appears quite
mann suggested an alternative denition of entropy in 1877 in frequently in textbooks on thermodynamics:
terms of the number of microstates comprising a given macro- (i) Mixing of two (or more) dierent ideal gases is a process
state. Boltzmanns denition opened the door to a molecular that causes an increase in disorder.
interpretation of entropy. Boltzmann stated:1 (ii) Increase in disorder results in an increase in entropy.
When this is the case, then whenever two or more small parts of (iii) It follows from i and ii that mixing of dierent ideal gases
it come into interaction with each other, the system formed by should be associated with an increase in entropy.
these parts is also initially in an ordered state, and when left to Statement iii, or some variations of it, appears in almost all
itself it rapidly proceeds to the disordered most probable state. textbooks of thermodynamics and statistical mechanics. As a
Note that Boltzmann used the terms disorder and probability consequence of this conclusion, most textbooks add the state-
almost as synonyms. Both of these concepts are related to the ment that mixing of dierent ideal gases is an inherently irreversible
number of microstates; the larger the number of microstates, the process.
larger is the disorder, and the larger is the probability of the state. The fallacy of this conclusion has been discussed in great detail
The interpretation of entropy as disorder has prevailed in the more than 20 years ago.5,6 Here, I demonstrate how the fallacy of
literature for over 100 years.2-4 It is true that many spontaneous this conclusion results from the association of entropy with
processes may be viewed as proceeding from an ordered to a disorder. I do this by means of a few examples shown in Figures
disordered state. However, there are two diculties with this 1-3. In all of these processes, we are not interested in the actual
interpretation. First, the concept of order is not well-dened, and process leading from the initial to the nal state, but only in the
in many processes it is dicult, if not impossible, to decide which dierence in the entropy of the system between the initial and the
of the two states of the system is more or less ordered.4-6 Some nal states. All the processes described may be performed quasi-
specic examples are provided in refs 4 and 6. Note however that statically at constant temperature.2 Because the systems contain
the term disorder here is used in its colloquial sense. There are only ideal gases, constant temperature also implies constant
some specic systems for which an order parameter may be average kinetic energy of the particles.
dened, but this is not the type of order discussed in describing
entropy. Second, there are cases in which an increase in disorder EXAMPLES
is clearly observed, yet the entropy change can be either zero or Consider the three processes shown in Figure 1. In these
negative, therefore, invalidating the interpretation of entropy in processes we start with N molecules of one kind and N molecules
terms of order and disorder (see below). In spite of these dif- of a second kind and end having the two dierent gases mixed in a
culties, the order-disorder interpretation still lingers in the volume V. Although there exists no precise denition of order or
literature.2,3
In this article, a few processes involving mixing and demixing
of ideal gases are presented. It is shown that the qualitative Published: March 07, 2011

Copyright r 2011 American Chemical Society and

Division of Chemical Education, Inc. 594 dx.doi.org/10.1021/ed100922x | J. Chem. Educ. 2011, 88, 594596
Journal of Chemical Education ARTICLE

Figure 2. Two processes involving positive change in entropy. In pro-

cess I, mixing occurs, but in process IV, demixing occurs.

Figure 1. Three processes of mixing. The nal states are the same in
these processes; however, the entropy changes are dierent.

disorder, we feel intuitively that the mixing of two things, whether

two dierent gases or mixing apples and oranges, is a disordering
process. Therefore, statement i is qualitatively and intuitively
accepted as correct.
Statement ii is commonly accepted as part of the interpreta-
tion of entropy as a measure of disorder. Even Gibbs used the
term mixedupness in connection with his analysis of the entropy Figure 3. A process, IV0 , in which almost complete demixing occurs
of mixing.7 Clearly, it is not possible to check the validity of the spontaneously.
second statement, qualitatively or quantitatively, because we do
not have a precise measure of order or disorder. Instead, we can
check the validity of statement iii and indirectly infer from it on SIV 2R ln 2 > 0 5
the validity of statement ii. To do this, we calculate the entropy
change either by classical thermodynamics or by statistical
mechanics for the three processes. The results are well-known,6
Process I can be carried out spontaneously simply by remov-
SI 2R ln 2 > 0 1 ing the partition separating the two gases. Process IV can be
performed by utilizing semipermeable partitions, in two steps;
one spontaneous, involving partial demixing with S = 2R ln 2,
SII 0 2 followed by a reversible demixing of the remaining mixture.3,5
Here, the terms reversible and irreversible are employed in the
sense used by Callen,2 that is, when the entropy of the system
SIII - 2R ln 2 < 0 3 does not change and when it increases, respectively. The term
quasi-static is used to describe a process carried out along an
where R is the gas constant. Note that in these processes, we are
innite sequence of equilibrium states.2
not concerned with the path leading from the initial to the nal
If one concludes, on the basis of process I, that mixing is a
state; we are only interested in is the dierence in entropy between
spontaneous irreversible process, one must also conclude from
the initial and the nal states.
the result in process IV, eq 5, that demixing is a spontaneous
Clearly, these results are in conict with statement iii, that is, in
irreversible process. Clearly, these two conclusions cannot be
all the processes in Figure 1 we observe mixing, but the entropy
both correct. In fact they are both wrong, neither mixing nor
change is either positive, zero, or negative. Because we have
demixing of ideal gases, in themselves, aect any thermodynamic
agreed that statement i is qualitatively and intuitively correct, we
quantity. Therefore, mixing was dubbed to be a thermodynamic
must conclude that statement ii is incorrect. Thus, we conclude
nonprocess.5
that increase in disorder cannot, in general, be associated with
Why do most textbooks on thermodynamics claim that mixing
increase in entropy. This association is sometimes correct, as for
is an irreversible process and that mixing involves increase in
process I, but incorrect for processes II and III (Figure 1).
entropy? The probable answer is that in daily life we see many
Before analyzing the psychological reasons for the commonly
mixing processes that are spontaneous. A drop of ink dropped
held view that associates mixing with disorder and before suggesting
into a glass of water will mix with the water, and we never see the
an alternative interpretation of the quantitative results in eqs 1-3,
reversal of these process. Processes similar to process I in Figure 2
let us discuss another conclusion often stated in textbooks on
are spontaneous and irreversible.
thermodynamics, namely that mixing of dierent ideal gases is an
Admittedly, demixing processes that occur spontaneously are
inherently spontaneous irreversible process.3,8 To see the fallacy of the
rare, but they do occur. If one vigorously mixes water with oil and
latter statement, consider the two processes I and IV shown in
let it settle, the two liquids will demix spontaneously. Another
Figure 2. For both of these processes the entropy change can be
example is shown in Figure 3 that is essentially the same as
calculated:
process IV. We start with mixture of A and B in a small volume v.
We then open two semipermeable partitions, one partition that
SI 2R ln 2 > 0 4 allows A to pass but not B, and the second partition that allows B
595 dx.doi.org/10.1021/ed100922x |J. Chem. Educ. 2011, 88, 594596
Journal of Chemical Education ARTICLE

to pass but not A. If the volume V is much larger than volume v, In all the processes discussed in this article, only the accessible
we shall observe a spontaneous irreversible demixing of most of volume for each particle has been changed. Because we are dealing
the mixture. The process of the demixing can be completed by with ideal gases, the change in volume does not involve a change
reversibly moving the two partitions toward each other.6 Note in the velocity distribution, that is, no change in the velocity
again that in the process of Figure 3, we are considering a (or momentum) information. The only change we observe is the
demixing of ideal gases. Such processes are rare. However, when change in the locational information.6,10
intermolecular interactions exist, one can devise many processes Qualitatively, having a particle in a box of volume V, we can
in which demixing occurs, for example see Chapter 6 in ref.6 always divide the box into M small cells each of size v. The SMI
If we show process IV0 to a student who has just learned that may be dened as the number of binary questions one needs to
mixing is an irreversible process, he or she will almost certainly ask to nd the location of the particle. It is easy to show that,
react by claiming that what we have done in process IV0 is simply although the SMI depends on the size of the cells, the change in
expansion of the gas from a small to a large volume and that this the SMI does not.5 What matters is only the change in the
expansion is the driving force for the irreversible process and the number of cells M, from the initial to the nal state. It can be
cause of the increase of entropy. The demixing has nothing to do shown that in the limit of very small size cells the change of the
with the increase of entropy. This explanation is correct. Indeed, SMI, similar to the change in entropy is proportional to the
the demixing in this process has nothing to do with the increase logarithm of the ratio of the volumes in the two states.6 In terms
in entropy; this is simply a process of expansion! However, this is of locational SMI, we can interpret the four processes in
also true for the mixing process denoted I in Figure 1. It is the Figures 1-3 as follows:
expansion rather than the mixing that drives process I, that is, the 1. In process I, the accessible volume per particle increases;
expansion of each gas from V to 2V. hence, the SMI increases.
Thus, the correct conclusion from the observation of processes I 2. In process II, the accessible volume per particle does not
and IV0 is not that the mixing or demixing is irreversible, but that in change; hence, no change in the SMI.
both processes the expansion is the driving force and the cause of the 3. In process III, the accessible volume per particle decreases;
increase in entropy. While in one process we observe mixing and in hence, the SMI decreases too.
the other process we observe demixing, neither mixing nor demixing 4. Similarly, in both processes I and IV0 , there is an increase in
has any eect on the entropy change in the process.5,6 SMI due to the increase in the accessible volume.
Gibbs,7 who probably was the rst scientist to analyze the Thus, in process I and IV0 the entropy increases, in process II the
thermodynamics of mixing, was apparently more puzzled by the entropy is constant, and in process III the entropy decreases. In
fact that the entropy of mixing of the two dierent gases is all of these processes, mixing is observed but the mixing is
independent of the type of gases, than by the fact that when mixing inconsequential. Therefore, we did not deem it necessary to
two gases of the same kind, there is no change in entropy. Yet it mention mixing, order, or disorder in the four statements
seems that Gibbs failed to see that in process I, which he referred to made above.
as a mixing of gases of dierent kinds, is simply an expansion
process and that the mixing in this process is only incidental.
Realizing that process I is just an expansion, there is no AUTHOR INFORMATION
puzzlement. The change in entropy associated with the expan-
sion process is independent of the kind of gas. The entropy Corresponding Author
change is simply the result of the increase in the accessible *E-mail: [email protected].
volume for each particle, from V to 2V. Clearly, this entropy
change must be independent of the type of gas that is expanding.
REFERENCES
MEASURE OF INFORMATION (1) Boltzmann, L. Lectures on Gas Theory; S. G..Brush, Translator;
University of California Press: Berkeley, CA, 1964; pp 442-443.
This brings us to the alternative interpretation of entropy (2) Callen, H.B. Thermodynamics and an Introduction to Thermo-
change in terms of Shannons measure of information.9,10 As we statics, 2nd ed.; John Wiley and Sons: New York, 1985.
have seen, the incorrect conclusion reached in statement iii is due (3) Denbigh, K.G. Principles of Chemical Equilibrium; Cambridge
to the association of entropy with disorder. This conclusion University Press: London, 1966.
would not have been reached had we associated the entropy with (4) Ben-Naim, A.; Entropy Demystied. The Second Law Reduced to
Shannon measure of information.6,9,10 The fact that the order- Plain Common Sense; World Scientic: Singapore, 2007.
disorder interpretation still lingers in the literature, in spite of the (5) Ben-Naim, A. Am. J. Phys. 1987, 55, 1105.
(6) Ben-Naim, A.; A Farewell to Entropy, Statistical Mechanics Based
superiority of the informational interpretation, results because
on Information; World Scientic: Singapore, 2008.
the concept of information is very general and encompasses both (7) Gibbs, J. W. Scientic Papers, Thermodynamics; Longmans Green:
subjective and objective types of information. Only one measure New York, 1906; Vol. I,
of the amount of information as dened by Shannon9 is relevant (8) Rowlinson, J.S.; Swinton, F. L. Liquids and Liquid Mixtures, 3rd
to the interpretation of the entropy.5,10 The Shannon measure of ed.; Butterworth: London, 1982.
information (SMI) is discussed in detail in ref 6. It is enough to (9) Shannon, C. E. A Mathematical Theory of Information. Bell Syst.
say here that, for all the processes described in this article, the Tech. J. 1948, 27, 379.
change in SMI is identical with the change in the thermodynamic (10) Ben-Naim, A. Discover Entropy and the Second Law of Thermo-
entropy, except for the multiplicative Boltzmann constant; that dynamics; World Scientic: Singapore, 2010.
is, the change in the SMI is simply ln(V2/V1), whereas the change
in entropy is kB ln(V2/V1). The informational interpretation of
the change in entropy is provided by the SMI.
596 dx.doi.org/10.1021/ed100922x |J. Chem. Educ. 2011, 88, 594596