Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Why is the letter "h" (or "H") used to denote entropy in information theory, ergodic theory, and physics (and possibly other places)?

Edit: I'm looking for an explanation of the original use of "H". As Ilmari Karonen points out, Shannon got "H" from Boltzmann's H-theorem. So (assuming Boltzmann actually used "H"), the original use is at least as early as that.

share|improve this question
2  
I don't know why, but Shannon used that letter in his foundational paper (page 10 in the linked pdf), and here's a scan of the original, page 392f. –  t.b. Nov 22 '11 at 21:24
    
@t.b.Thanks. I'm hoping for an explanation of the original use. –  Quinn Culver Nov 22 '11 at 21:36
    
In relation with what t.b. said, the link between information theory entropy and physical entropy as introduced by boltzmann was not initially understood by shannon. it was von neumann who advised him to call this "entropy". –  Glougloubarbaki Nov 22 '11 at 21:44
    
According to this letter to nature (screen shot here, Boltzmann actually used $E$, not $H$. It appears that there is some contention about your question, the German Wikipedia entry on the $H$-theorem lists two further papers (in English) concerning this question. –  t.b. Nov 22 '11 at 22:19
add comment

2 Answers 2

up vote 9 down vote accepted

Wikipedia claims, citing "Gleick 2011", that Shannon got the letter $H$ from Boltzmann's H-theorem. Indeed, Shannon writes in his 1948 paper on page 393, after defining $H = -K \sum_{i=1}^n p_i \log p_i$:

"The form of $H$ will be recognized as that of entropy as defined in certain formulations of statistical mechanics8 where $p_i$ is the probability of a system being in cell $i$ of its phase space. $H$ is then, for example, the $H$ in Boltzmann's famous $H$ theorem."

Of course, this just changes the question to "Why did Boltzmann choose the letter $H$, then?" In this letter to the editor, published in Nature in 1937, Sydney Chapman writes:

"WHEN Boltzmann first published the celebrated theorem now generally known as the $H$-theorem, he used the symbol $E$ (presumably as the first letter of entropy), not $H$. It has been suggested that when $H$ was first used for this theorem it was intended to be the capital Greek letter eta: but the first paper known to me in which $H$ is used for Boltzmann's entropy function is one by Burbury1, who seems to have changed Boltzmann's symbol $E$ to $H$ for no special reason; later Burbury used $B$ for an almost identical function, which he called Boltzmann's minimum function2. Boltzmann himself wrote $E$ so late as 18933, but in 18954 he used the letter $H$. This use of $H$ must have seemed mysterious to many generations of students, and it would be interesting to know whether any reader can account for its use or give an earlier instance of it."

So apparently, you're far from the first person to wonder about this.

Indeed (thanks to t.b. for the links), 30 years later, in a letter to the Americal Journal of Physics, Stephen G. Brush repeated Chapman's plea, and added that "Professor Chapman informed me, a couple of years ago, that he never received any response to this letter." 10 years later yet, in the same journal, Stig Hjalmars wrote in response to Brush's letter:

"The given graphical evidence, of which a detailed account is presented elsewhere,6 seems to leave no reasonable doubt that that during the decade before Boltzmann's death in 1906 at least he himself, Gibbs and Zermelo meant a capital eta when they wrote $H$ for Boltzmann's function."

The cited "elsewhere" is "S. Hjalmars, TRITA-MEK-76-01, Technical Reports from the Royal Institute of Technology, Department of Mechanics, S-10044 Stockholm, Sweden," stated to be "Free of cost on request from the Department." Alas, I haven't so far managed to locate a copy of this report.

share|improve this answer
    
Yes, it does just change the question. I'll edit accordingly. Thanks though. –  Quinn Culver Nov 22 '11 at 22:05
    
The connection between H and capital eta seems most plausible. Thanks. –  Quinn Culver Nov 22 '11 at 22:26
    
The etymology given in OED suggests that "trope" is Greek for "turing", and I wonder if that means "turning" in the sense of available energy "turning into" unavailable energy. It goes on to suggest that the initial "en-" was intended to parallel the "en-" in "energy", which is derived in part from "erg" which is related to "work". I thought it was a useful mnemonic to think that "H" is the capital "eta", but the etymology says the initial "e" actually comes from epsilon, not eta. So maybe it's a useful mnemonic. Except that I never needed a mnemonic for this. –  Michael Hardy Nov 23 '11 at 4:08
add comment

From the wikipedia, Ralph Hartley in 1928 wrote H = n log s for the amount of information transmitted, with s the number of possible symbols. But I'm not sure why he used H.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.