Tutorials Developer

Claude Shannon: The Bit, and the Limits of Any Channel

By Infinite Creation · Profile

Historical photograph of Claude Shannon.

At a Glance

Claude Elwood Shannon (1916-2001) published "A Mathematical Theory of Communication" in 1948, founding information theory. It defined the bit, established entropy as the measure of information, set the maximum rate any noisy channel can carry, and proved that coding can achieve reliable transmission up to that limit. Every codec, modem and storage system in this corpus operates inside his results.

Lived: 1916-2001, United States
Anchor year: 1948, A Mathematical Theory of Communication
Strand: canon

Career

The problem as it stood

Engineers had rules of thumb about bandwidth, noise and signal power but no theory of what information *is* or of the fundamental limits. Nobody could say whether a proposed system was close to optimal or hopelessly wasteful.

What he actually did

Shannon defined information in terms of surprise, the less predictable a message, the more information it carries, measured it in bits, and derived the capacity of a channel from its bandwidth and signal-to-noise ratio. He then proved that codes exist achieving error-free transmission at any rate below capacity.

How it worked

Entropy measures the average unpredictability of a source, and it sets the floor for lossless compression: no scheme can beat it. Channel capacity sets the ceiling for transmission. Together they bracket everything: compression cannot go below entropy, and transmission cannot exceed capacity, regardless of cleverness.

What it made possible

Every compression scheme, error-correcting code, modem and digital broadcasting standard. It also reframed the whole field: after 1948 it became possible to ask whether a design was near optimal rather than merely whether it worked.

What happened to him

He worked at Bell Labs and MIT, also founding digital circuit design with his 1937 master's thesis applying Boolean algebra to switching. He built juggling machines, a mechanical mouse and a flame-throwing trumpet, and was famously indifferent to promoting his own work.

Where the credit landed

Shannon is properly and thoroughly credited, and remains far less publicly known than his importance warrants. His 1937 thesis, arguably as consequential as the 1948 paper, is barely mentioned outside computing. Both were done at institutions. Bell Labs and MIT, that took the patents.

Known For

Entropy is the floor for lossless compression. Channel capacity is the ceiling for transmission.
Information is surprise: predictable messages carry less of it.
His 1937 master's thesis founded digital circuit design.

Knowledge Base

What You'll Learn

After 1948 you could ask whether a communication system was close to the best possible.

Lossless, lossy, and where entropy stops applying

Entropy bounds lossless compression: you cannot represent a source in fewer bits than its entropy and still recover it exactly. Lossy compression escapes the bound by not recovering it exactly: MP3 and JPEG discard content judged imperceptible, so they are not beating Shannon but changing the problem. Rate-distortion theory, also Shannon's, describes that trade formally: how little data suffices for a given tolerated error.

Where This Fits

Shannon set the limits of every transmission link covered in A History of Broadcast Technology.

FAQ

Q: What is information entropy?
A: A measure of the average unpredictability of a source. Highly predictable data has low entropy and compresses well. Random data has high entropy and cannot be compressed losslessly.

Q: What is channel capacity?
A: The maximum rate at which information can be transmitted over a noisy channel with arbitrarily low error, determined by bandwidth and signal-to-noise ratio. Shannon proved codes exist that approach it.

Vocabulary

Bit. The smallest unit of information, representing a choice between two equally likely possibilities. The common currency that lets text, audio and video be measured, compared and compressed on the same scale.

Entropy. A measure of how unpredictable a source of information is. High entropy means little redundancy and little to compress. Low entropy means the data repeats in ways a coder can exploit.

Channel capacity. The maximum rate at which information can pass through a channel with arbitrarily small error, given its bandwidth and noise. Sets a hard ceiling no coding scheme can exceed.

Noise. Unwanted variation added to a signal in transmission or storage. The reason channel capacity is finite, and the quantity every error-correction scheme exists to work against.

Sources

The founding paper is available as a scan of the journal issue it appeared in. Checked 20 August 2026.

A Mathematical Theory of Communication, Bell System Technical Journal, volume 27 number 3, July 1948, pages 379 to 423. The first half of the paper, scanned from the original issue. It introduces the bit as a unit, entropy as a measure of information, and the treatment of a communication channel as having a capacity.
A Mathematical Theory of Communication, Bell System Technical Journal, volume 27 number 4, October 1948, pages 623 to 656. The second half, published three months later in the same journal, which extends the treatment to continuous signals.