Dead Reckonings

An operational calculus converts derivatives and integrals to operators that act on functions, and by doing so ordinary and partial linear differential equations can be reduced to purely algebraic equations that are much easier to solve. There have been a number of operator methods created as far back as Leibniz, and some operators such as the Dirac delta function created controversy at the time among mathematicians, but no one wielded operators with as much flair and abandon over the objections of mathematicians as Oliver Heaviside, the reclusive physicist and pioneer of electromagnetic theory.

The name of Oliver Heaviside (1850-1925) is not well-known to the general public today. However, it was Heaviside, for example, who developed Maxwell’s electromagnetic equations into the four vector calculus equations in two unknowns that we are familiar with today; Maxwell left them as 20 equations in 20 unknowns expressed as quaternions, a once-popular mathematical system currently experiencing a revival for fast coordinate transformations in video games. Heaviside also did important early work in long-distance telegraphy and telephony, introducing induction-loading of long cables to minimize distortion and patenting the coaxial cable. At one time the ionosphere was called the Heaviside layer after his suggestion (and that of Arthur Kennelly) that a layer of charged ions in the upper atmosphere (now just one layer of the ionosphere) would account for the puzzlingly long distances that radio waves traveled. But Heaviside was an iconoclast who saw little need to ingratiate himself with others or spend time justifying his methods to them. It took later mathematicians such as Carson and Bromwich to demonstrate that his operators are analogous to later, well-developed integral equations and contour integrals in the complex plane.

Actually, absence of rigor is less unusual historically than it might appear—much mathematical science has progressed on very shaky ground indeed, and often proofs of mathematical techniques lag by many years their application. The famous mathematician G. Hardy wrote

All physicists and a good many quite respectable mathematicians are contemptuous about proof.

and certainly mathematicians often discover things by intuition that require a great deal of time and labor to prove. Often mathematics progresses from the specific to the general, bottom to top. Carl Friedrich Gauss once wrote in his diary about one of his most important discoveries, which was based on a purely computational observation: “I have the result, but I do not yet know how to get it.” The mathematician J. Hadamard observed, “The object of mathematical rigor is to sanction and legitimize the conquests of intuition, and there was never any other object for it.”

Today “experimental mathematics”—using computers to search for number-theoretic results that can be generalized—is a hot field in mathematics. As David Berlinski writes,

The computer has in turn changed the very nature of mathematical experience, suggesting for the first time that mathematics, like physics, may yet become an empirical discipline, a place where things are discovered because they are seen.

And even back in 1951, Kurt Godel wrote,

If mathematics describes an objective world just like physics, there is no reason why inductive methods should not be applied in mathematics just the same as in physics.

Heaviside was openly dismissive of attempts to provide rigor for his operator calculus. Here’s what he had to say regarding his bold generalization of an experimental result to a general one that we will discuss shortly:

Those who may prefer a more formal and logically-arranged treatment may seek it elsewhere, and find it if they can; or else go and do it themselves.

Perhaps his refusal to validate it was a good thing because his operator calculus was not generally rigorous, particularly when initial values are non-zero. Heaviside used a great deal of intuition to guide him in the process of applying his calculus. In the end Laplace transforms, easier to use with a more rigorous structure and incorporating the powerful tool of convolution, overtook the operational calculus of Heaviside, and his methods largely fell victim to history. But the ingenuity of Heaviside appeals to me (along with his sheer indifference to the complaints of those he left in the dust) and so this essay is a short appreciation of his work in this regard.

Let’s first get a flavor of how his operators worked. Heaviside took the basic equations for voltage v and current i for a discrete resistance R, capacitance C and inductance L and rewrote them using his operator p, which performed the derivative with respect to time on the function to the right of it (effectively d/dt), as in p·f(t). He also assumed that the inverse of p, or 1/p, is the operator performing the integral of a function, so that p · 1/p = 1/p · p = 1. This is not generally true because when the integral is taken after the derivative there is a constant term, but it is true for a function where f(0)=0. So we have

v  =  iR
i  =  Cpv     from v = (1/C)  ∫ i dt
v  =  Lpi      from v = L di/dt

Then he separated the operators from their functions, giving them an independent existence, to form a general AC impedance defined as v/i that he called Z (a term we still use today):

Z  =  R
Z  =  1 / (Cp)
Z  =  Lp

Heaviside generally treated problems in which a constant voltage is applied to a circuit at time t=0, in other words an impulse (or step function) such as might be encountered as transient signals on cables. In telegraphy these transient effects limit the signaling speed, while in telephony the transient effects limit the line length. But this step function also has the property that f(0) can be treated as 0, the condition for the commutivity of the inverse operator given above. Using the superposition property from this waveform also allowed him to analyze distortion at all signal frequencies. In fact, Josephs remarks that, explicit or not, it is always assumed in the differential equations for transient currents that the applied voltage is 0 for t<0. Heaviside wrote this step function as the bold symbol 1. So if we assume we have a circuit with a resistance and inductance,

Z  =  R + Lp

Then for v = 1 here,

i  =  v/Z

   =  1 / (R+Lp)

   =  (1/R)  [1 / (1 + Lp/R)]

   =  (1/R)  [R/(Lp) / (R/(Lp) + 1)]

   =  (1/R)  [R/(Lp)]  [1 / (1 + R/(Lp))]

   =  (1/R)  [R/(Lp)]  [1 – R/(Lp) + (R/(Lp))² – (R/(Lp))³ + · · ·]    by the Binomial Theorem

   =  (1/R) [(R/L) · 1/p – (R/L)² · 1/p² + (R/L)³ · 1/p³ – · · ·]

Now for the step function 1, the “integral” 1/p · 1 = t, and in general 1/pⁿ · 1 = tⁿ/n! , so

i  =  1/R [R/L · t – (R/L)² · t²/2! + (R/L)³ · t³/3! – · · ·]

But the power series expansion of e^-t is

e^-t  =  1 + t + t²/2! + t³/3! + · · ·

so i can be rewritten as:

i  =  1/R [1 – e^-(R/L)t]

yielding the correct exponential rise of current through the circuit with a time constant of 1/(R/L) = L/R.

To solve the differential equation y” – y = 0 for t>0 and y(0)= 0 and y'(0) = 0, Heaviside would rewrite this as p²y – y = 1, or y = 1 / (p² –1). But 1/(p² – 1) can be expanded into the series (1/p² + 1/p⁴ + 1/p⁶ + · · · ), and since we have 1/pⁿ · 1 = tⁿ/n! from above, then y = t²/2! + t⁴/4! + t⁶/6! + · · ·  =  ½(e^t – e^-t) – 1. You can see how easily one can get in the habit of dropping the 1 altogether and working solely with p’s once the knack is acquired.

As another example, Heaviside presented an operator function [p/(p+B)]^1/2 acting again on a step function. Here he divides through by p to get (1 + B/p)^-1/2 and again expands this into a power series, arriving at a solution in terms of modified Bessel functions. Can p’s be handled in these ways? Apparently they can, for Heaviside was able to provide correct solutions using his operator calculus.

Now in circuits with continuous, distributed impedances such as in telegraph lines, (particularly distortionless ones in which an infinite current appears instantaneously at t=0) Heaviside was faced with fractional powers of p such as p^1/2. Unfazed, he found a specific problem that had been solved using Fourier series methods (the solution of the diffusion equation) and applied it to a problem for which p^1/2 appeared when written in his operator form (the current from a step voltage in an infinitely long cable). Heaviside equated the form of his solution to the Fourier solution to deduce p^1/2 and he declared the result to be generally true! (Later, Heaviside presented a direct derivation based on the gamma function, but it is also derivable using Carson integrals and other methods).

He then arrived at his fractional powers of p:

    p² 1  =  1 / (πt)^1/2

 p^3/2 1  =  p(p^1/2 1) = – t^-3/2 / 2π^1/2

p^-1/2 1  =  t^1/2 / (1/2)!

p^-3/2 1  =  1/p (p^-1/2) 1

and so forth.

Heaviside developed the Heaviside Expansion Theorem to convert Z into partial fractions to simplify his work. For i=1/Z and Z a polynomial in p, the roots of Z can be found and i expressed as a sum of terms consisting of constants divided by the simpler factors. A similar thing is done when using Laplace Transforms, but Heaviside developed his own method of calculating these constants, the Heaviside Cover-Up Method (See this webpage for a description).

Heaviside expressed the use of this theorem for a step function as

i  =  v/Z₀ + v ∑ e^pt / [p(dZ/dp)]

So let’s look again at our first problem above of a series circuit of a resistance and an inductance, where Z = R + Lp and the steady state Z0 = R. Now dZ/dp = L and setting R+Lp = 0 implies a root p = – R/L. Then from the equation just above after setting the step value v to 1:

i  =  1/R + e^-(R/L)t / [(–R/L) · L]

or

i  =  1/R [1 – e^-(R/L)t]

which is the same solution we obtained earlier using power series.

Nahin describes Heaviside’s ingenious solutions when using his operators for the time-varying current in circuits with additional continuously-distributed parameters such as found in actual telegraph lines. He provides an example in which Heaviside adds a section of cable to the beginning of an infinite line, finds the current as a function of time for that configuration, and then “removes” the initial section to end up with the solution for the original cable. In fact, Heaviside used his operator calculus to design a transmission line with zero distortion (but with exponential attenuation over distance).

Now when Z is a polynomial of degree greater than 4, its roots are difficult or impossible to find directly. Also, the Expansion Theorem does not work for a Z with a root of zero or with any repeated roots, a situation not encountered in passive networks but one that can occur when an internal component such as an amplifier sources energy. In true Heaviside fashion we can in this case treat equal roots as unequal, solve for the transient current, and let the roots approach equality as a limit!

But Heaviside removed even these difficulties by instead expanding Z in inverse powers of p and then replacing p by tⁿ/n! exactly as we saw in our first example. I’ve seen this referred to as Heaviside’s Extended Expansion Theorem, and I really have to wonder whether punchier names for his methods would have preserved more interest in them. Heaviside, who coined some terms much disliked in his day but which have stuck (such as impedance, inductance, conductance, admittance and reluctance), awkwardly referred to “algebrising” a differential equation with his operators and “logarising” when taking a logarithm, and he christened his operator e^-ph the “Spotting function” because it isolates, or spots, a certain value of the function. Along with his “Cover-Up Method” these are not exactly memorable names.

Now as one more example of this method, consider a series circuit of R, L and C, where Q represents the charge in the circuit, so the current i = dQ/dt = p·Q:

Z  =  R + pL + 1/pC

i  =  1 / [R + pL + 1/pC]

[R + pL + 1/pC] i  =  1

[p²L + pR + 1/C] Q  =  1

Q  =  1 / [p²L + pR + 1/C]

=  1 { p²L · [1 + (R/L)p^-1 + (1/LC) p^-2]^-1}

Expanding the term on the right using the Binomial Theorem, we have

Q  =  (1/L) · {p^-2 · [1 – [(R/L)p^-1 + (1/LC)p^-2] + [(R/L)p^-1 + (1/LC)p^-2]² – · · · }

     =  (1/L) · {p^-2 – (R/L)p^-3 – [(1/LC) + R²/L²]p^-4 + · · · }

and as before we replace p^-n · 1 by tⁿ/n! to arrive at a power series in t without having to find the roots of Z. These power series are sometimes not expressible in terms of elementary functions, but Heaviside dismissed this with the astute observation that calling the sum of a particular power series an exponential or trigonometric function didn’t simplify finding the original solution of it.

In 1893 Heaviside published the first of a three-part series describing his operator calculus in the Proceedings of the Royal Society. Later in the year the second part appeared, but this “was the last straw for mathematicians” [Nahin], and his third part was rejected, leading to a series of written attacks best enjoyed by reading Nahin’s book.

What killed the third part was Heaviside’s cavalier use of divergent series, dismissing their apparent tendencies to infinity while producing accurate results by manipulating them at whim, or at least this was how it appeared. In fact, Heaviside often produced two versions of his power series solution, a convergent one that was useful for small t but was too slow to converge for large t, and a divergent one that was useful for large t when it was taken to a small number of terms. Josephs provides an example of two such series for the Bessel function solution of a current entering a particular type of transmission line:

e^-φtI₀(φt)  =  1 – (φt) + (1·3/(2!)²) (φt)² – (1·3·5/(3!)²) (φt)³ + · · ·

e^-φtI₀(φt)  =  (2πφt)^-1/2 [ 1 + (8φt)^-1 + 1²·3²/(2!) · (8φt)^-2 + 1³·3³·5³/(3!) · (8φt)^-3 + · · · ]

The first expression is convergent but is slow to converge for larger values of φt, while the second expression is a divergent series of an asymptotic type (a series in inverse powers of the argument). The plot below shows the effect of taking terms in each series for relatively large values of φt. The first expression is shown as convergent, but slowly, while the second expression converges quickly to approximately the final value of the convergent expression as the values of the terms decrease, then takes off and rises to much larger values. The larger the value of t, the less is the error at the point of the minimum term. The trick is to stop taking terms after the one with the smallest value, which Heaviside could do empirically.

In general, Heaviside found that if an operational equation is a series in integral and fractional powers of p, he could discard the terms with integral powers of p, express the fractional powers of p as pⁿ p^1/2 or pⁿ (πt)^-1/2, and create a divergent series in pⁿ useful for large t. Josephs provides an example of this procedure in deriving a convergent and divergent series for the voltage at the terminals of a non-inductive cable when a 1-Volt battery is applied through a terminal resistance.

Heaviside used physical intuition to guide him in handling these series, and he was unparalleled in his electromagnetic intuition. And of course he went far beyond the short flavor of his operator calculus I’ve described here—Heaviside found solutions for very complicated electromagnetic problems whose solutions were intractable by any other method at the time.

But Heaviside also checked his steps with exhausting numerical calculations to make sure he didn’t make a misstep. I really appreciate this because I have often done that in deriving approximations to elementary functions, an interest of mine apparent on my MyReckonings.com website—I was really gratified to find a professional doing something like this. Heaviside also took his solution and verified that it satisfied the original equation, although of course this doesn’t validate the uniqueness of it. Because Heaviside’s work was results-oriented, he sometimes provided ad hoc arguments to support his derivations. He often suggested in rather abrupt prose that mathematicians should provide rigorous proofs for what he did. I see a bit of his humanity in a footnote in Volume 2 of his Electromagnetic Theory (a volume almost exclusively concerned with his operator methods):

It is rather disagreeable to have to be self-assertive and dogmatic (especially when one thinks of the always possible risk of error); but there may be times when it becomes a duty—e.g., when mathematical rigourists are obstructive.

And Heaviside certainly did exercise the option, often with a Victorian flair for the erudite put-down:

I think I have given sufficient information to enable any competent person to follow up the matter in more detail if it is thought to be desirable. It is obvious that the methods of the professedly rigorous mathematicians are sadly lacking in demonstrativeness as well as in comprehensiveness.

Mathematicians were not amused, but I am.

References

Heaviside, Oliver. Electromagnetic Theory, Vols. 1-3. New York: Cosimo Classics (2007). These three volumes, available at Amazon, are reprints of Heaviside’s books on electromagnetic theory. Volume 2 contains a modified version of the rejected Part III of his series on operators for the Proceedings of the Royal Society. The three volumes were also printed as one large book by Dover in 1950.

Heaviside, Oliver. On Operators in Physical Mathematics, Part I. Proceedings of the Royal Society, vol. 52, Feb., 1893, pp. 504-529. Online scans of this can be found at http://gallica.bnf.fr/ark:/12148/bpt6k56145p/f512.item and its following page selections.

Heaviside, Oliver. On Operators in Physical Mathematics, Part II. Proceedings of the Royal Society, vol. 52, Feb., 1893, pp. 504-529. Online scans of this can be found at http://gallica.bnf.fr/ark:/12148/bpt6k56147c/f112.item and its following page selections.

Josephs, H.J. Heaviside’s Electric Circuit Theory, 2nd Ed. Methuen’s Monographs on Physical Subjects, London: Methuen and New York: Wiley (1950). This tiny book (without the cover I measure it at 6-5/8″ x 4-1/8″ x ¼”) is packed with useful information on Heaviside’s operational calculus.

Lindell, I.V. Heaviside Operational Rules Applicable to Electromagnetic Problems. Progress in Electromagnetics Research, PIER 26 (2000), pp. 293-331, also found at http://ceta.mit.edu/PIER/pier26/11.9909172jp.Lindell.pdf. A comprehensive, highly mathematical collection of valid rules for Heaviside’s operator calculus along with their derivations.

Nahin, Paul J. Oliver Heaviside: The Life, Work, and Times of an Electrical Genius of the Victorian Age. Baltimore: Johns Hopkins University Press (1988). This is one of my favorite books of all time, a fascinating biography of Heaviside and his contemporaries and brimming with the controversies he engendered. It has a technical bent as well, as do all of Nahin’s excellent books.

Printer-friendly PDF of this post.