The Origin Story of the Sampling Theorem and Vladimir Kotelnikov

“The outstanding hero of the present. His merits are recognized all over the world. In front of us is the giant of radio engineering thought, who has made the most significant contribution to media communication development.” These words of praise for Vladimir Kotelnikov were spoken in 2000 by Bruce Eisenstein, president of IEEE. The main reason for such adoring words? A Russian conference report from 1933, which contained a rigorous proof of the remarkable sampling theorem.

The Sampling Theorem

The sampling theorem proves that an analog signal can be retrieved without errors and distortions from the sampling values — and outlines how this is done. The number of sampling values obtained per second must be at least twice as great as the highest frequency occurring in the original signal.

The theorem is fundamental to the design and analysis of today’s electronic communication systems. In an age of the internet, globalization, technical progress, and online preprints, it is hard to imagine that less than one hundred years ago, we were missing the scientific work that first addressed this problem.

The 15-year delay was due to several sad coincidences, language barriers, conservative publishing, and a gap between radiophysics and mathematics.

Here, we highlight some of the key moments from this “war of the currents“.

Miner, Whittaker, Nyquist, Kotelnikov, Shannon, Someya, et al.

At 24, Vladimir Kotelnikov was completing his postgraduate studies at Moscow Power Engineering Institute, where he became involved with several radio equipment tuning and development projects. This is also where Kotelnikov first experienced fundamental radio communication problems. In 1932, he prepared an extended and rigorous report, titled “On the Capacity of the ‘Ether’ and Cables in Electrical Communication”. He intended to present the report at a conference for the former Soviet Union.

Kotelnikov’s work contained not only theoretical analyses and the formulation of the theorem but also several important recommendations:

• Keep developing single-sideband technologies for signal transmission
• Investigate the use of directional antennas
• Extend the radio spectrum to ultrashort wavelengths
• Improve the frequency stability of radio transmitters

Single-sideband modulation theory.

Unfortunately, the conference where Kotelnikov planned to present his findings never happened. The paper, together with other symposium works, was published in 1933, but didn’t gain any attention among the scientific community. In 1936, Kotelnikov tried to republish the paper in a popular journal, Elektrichestvo, but it was rejected by the editorial board. They thought the paper’s topic was too narrow to interest readers. Therefore, Kotelnikov’s insights were concealed from the world for more than a decade.

Preliminary thoughts about sampling limits and the best practices of signal transmission can be found in several American and German engineering articles and patents; e.g., Miner (1903). A similar theorem was known in the mathematical community from the beginning of the 20^th century. It was proved by Whittaker in 1915 but without any real indications to radio applications.

In American literature, you can find references to Nyquist (1928), though it wasn’t explicitly related to analog signals. In Japanese articles, there are references to Someya (1949) that also discussed sampling. It wasn’t until 1949 that the sampling theorem was rediscovered by Claude Shannon. In his article “Communication in the Presence of Noise”, Shannon suggested that sampling was “common knowledge in the communication art” but somehow wasn’t formulated “explicitly in the literature of communication theory.”

These contributions finally became of great importance in the 1970s with the remarkable progress taking place in digital communication systems. After sorting out priorities, the scientific community decided to name the sampling theorem “WKS-theorem” in honor of Wittaker, Kotelnikov, and Shannon. In 1999, Kotelnikov was awarded the International 1999 Eduard Rhein Prize for his pioneering proof on the subject.

In 1999, researcher Hans Dieter Lüke made an exhaustive investigation of the sampling theorem’s origins, writing: “This history also reveals a process which is often apparent in theoretical problems in technology or physics: First the practicians put forward a rule of thumb, then the theoreticians develop the general solution, and finally someone discovers that the mathematicians have long since solved the mathematical problem which it contains, but in ‘splendid isolation.'”

Vladimir Kotelnikov’s Scientific Contributions

Any young researcher will find motivation in the fact that all of the underestimation of obtained results, lack of publication, etc. didn’t influence the scientific career of Vladimir Kotelnikov. In the 1930s, he initiated and successfully finished the development of a unique radio channel between Moscow and Khabarovsk (8615-km long!) based on single side-band techniques. During WWII, he was involved in scrambling and cryptography projects. After 1959, Kotelnikov became well known among theoreticians of communication technology worldwide through his monograph “The Theory of Optimum Noise Immunity” based on his doctoral thesis.

Kotelnikov made numerous contributions to radar and telemeter system development, statistical radiophysics; micro-, opto-, acousto- and magneto-electronics; and super- and semiconductivity. He was also a supervisor of interplanetary research and radiolocation for planets including the Moon, Venus, Mercury, Mars, and Jupiter.

A radar topography map of Venus. Image by Zamonin — Own work. Licensed under CC BY-SA 3.0, via Wikimedia Commons.

Transmissions from a Lasting Legacy

Vladimir Kotelnikov was known as a great supervisor and charismatic leader. He directed the Institute of Radio Engineering and Electronics in Moscow starting in 1953, acting as the Director Emeritus from 1987 to 2005. He was a member of the Russian Academy of Sciences and held the position of vice president from 1969 to 1989, coordinating dedicated academic research on a nationwide scale.

A photograph of Vladimir Kotelnikov (front row, second from left) in 1975, signing the Apollo-Soyuz Test Project (ASTP) agreement in Moscow. Image by NASA on The Commons. This file is in the public domain in the United States because it was solely created by NASA. NASA copyright policy states that “NASA material is not protected by copyright unless noted”.. Image via Wikimedia Commons.

Kotelnikov received numerous national honorary titles and international awards over the course of his life, such as the Hernand and Sosthenes Behn Prize from the IEEE, Theodore Von Kármán Prize from the International Academy of Astronautics, and IEEE Alexander Graham Bell Medal. He participated extensively in various international radiophysics and astronautics associations, including IEEE, and was the founder and chief editor of the Journal of Communication Technology and Electronics.

To complete the picture, over a number of years, Kotelnikov was involved in scientific and engineering personnel training and teaching as a professor and head of several university departments at the Moscow Power Engineering Institute and Moscow Institute of Physics and Technology.

Sculpture of Vladimir Kotelnikov in Kazan, Russia. Image by Анатолий Терентьев — Own work. Licensed under CC BY-SA 3.0, via Wikimedia Commons.

Vladimir Kotelnikov died in 2005 at 97 years old. He was still working on a monograph about quantum mechanics, his hobby throughout his career.

If you read any Soviet-era or Russian articles, book, or report on radiophysics, you’ll find references only to “the Kotelnikov theorem”.

Today, join us in wishing Vladimir Kotelnikov a happy birthday!

Applying Kotelnikov’s Work to Electromagnetic Modeling

The sampling theorem and many of Vladimir Kotelnikov’s other contributions are keystones of information theory, signal processing, and radiophysics in general. For example, during RF and microwave modeling, we often use them (or their consequences) while analyzing the time-domain reflectometry of microwave devices, performing RF signal transformation between the time and frequency domains or optimizing equipment for 5G networks.

References

1. Mark Bykhovskiy, “The Life Filled with Cognition and Action” IEEE Information Theory Society Newsletter, vol. 59, no. 3, pp. 13–15, 2009.
2. Vladimir Kotelnikov, “On the transmission capacity of ’ether’ and wire in electric communications“, Advances in Physical Sciences, vol. 49, pp. 736–744, 2006.
3. Neon Armand, “V.A. Kotelnikov and his role in the development of radiophysics and radio engineering“, Advances in Physical Sciences, vol. 49, pp. 744–748, 2006.
4. Hans Dieter Lüke, “The Origins of the Sampling Theorem“, IEEE Communications Magazine, vol. 37, no. 4, pp.106–108, 1999.
5. Chris Bissell, “Vladimir Aleksandrovich Kotelnikov: Pioneer of the sampling theorem, cryptography, optimal detection, planetary mapping“, IEEE Communications Magazine, pp. 24–32, 2009.