Dave Smith on Fifty Years of Synthesis: The 2016 Heyser Lecture

What This Document Is

On 29 September 2016, Dave Smith gave the 35th Richard C. Heyser Memorial Lecture at the Audio Engineering Society’s 141st Convention in Los Angeles. He called it Synthesizers: From Analog to Digital to Software to Analog, and he was unusually well placed to give it: he had designed the Prophet-5, proposed the interface that became MIDI, built one of the first commercial software synthesizers, and then walked away from software to make hardware again. This page is a close reading of that lecture, what he said, what the documentary record confirms, and the handful of places where the two do not quite line up.

It is worth being clear about why a page like this exists separately from the site’s history of the synthesizer. That page is a survey: it was written afterwards, from secondary sources, and it tells you what happened. This one starts from what a participant said about his own decisions, in his own framing, and treats that testimony as evidence to be weighed rather than as fact to be repeated. First-hand accounts are the most valuable kind of source and the least reliable kind of source at the same time, and both halves of that sentence matter.

On the source text. The transcript this was built from is machine-generated speech recognition, and it is bad. It renders Don Buchla as “Don Buckl”, Time Delay Spectrometry as “Time to Lisic Troometry”, Eurorack as “year Iraq”, and MIDI itself as “MIT”. Nothing on this page is presented as a verbatim quotation, because the transcript cannot support one. Where Smith’s distinctive phrasing is reproduced, it is marked as transcribed wording and should be checked against the AES recording before being quoted anywhere that matters. Every name, date and production figure here was verified against an independent published source.

The lecture, and the man it was named for

Richard C. Heyser (1931 to 1987) spent his career at Caltech’s Jet Propulsion Laboratory, where he worked on instrumentation for American space programmes from Explorer I onward. In 1967 the Journal of the Audio Engineering Society published his paper Acoustical Measurements by Time Delay Spectrometry, which showed how loudspeakers could be measured accurately in ordinary rooms rather than in an anechoic chamber. He received the AES Silver Medal in 1983, held nine patents, and was President-elect of the AES when he died on 14 March 1987.

The lecture series that carries his name was established in 1999 and funded through the Heyser Scholarship Fund. Two details from the introduction are worth recording because they are the kind of thing that survives only in spoken accounts. The first is Heyser’s own explanation of why he invented TDS at all: he was designing amplifiers as a hobby, could not justify the cost of an anechoic chamber, and turned to electronics to get around the expense, and then found, in the transcribed phrasing, that he had a tiger by the tail, because he could now measure things the acousticians did not seem to know about. The second is that the 1966 manuscript was reportedly thrown out at the AES and rescued from the bin by an anonymous reviewer who insisted it be published in full. That reviewer was named as Harry Olson, the RCA engineer. Heyser’s papers, including a great deal of handwritten material, are held at Columbia College Chicago.

Smith’s lecture also sits at a specific moment. Don Buchla had died on 14 September 2016, a fortnight earlier, and Smith opens his account of the 1960s by noting it. Smith himself died on 31 May 2022. The lecture is now the recollection of a man who is gone, about a man who had just died.

Two inventions, and the keyboard that divided them

Smith begins deliberately in the mid-1960s, setting aside Theremin, Max Mathews at Bell Labs, and everything earlier. His starting point is that voltage-controlled subtractive synthesis arrived twice, more or less simultaneously and independently: Bob Moog on the East Coast, Don Buchla on the West. Subtractive synthesis itself he describes as basic: start with a harmonically rich waveform, then shape it with filters, envelopes and amplifiers.

His substantive point is that the difference between the two men was not circuit design but the interface. Moog put a keyboard on his instrument. Buchla thought a keyboard was too restrictive, that it constrained what the technology could be. Smith argues this decision propagated directly into the music: Morton Subotnick’s Silver Apples of the Moon (1967) was made on a Buchla, Wendy Carlos’s Switched-On Bach (1968) on a Moog, and each piece belongs to the instrument it was made on. Carlos could not have played Bach without a keyboard. The Buchla record could not have been made on a keyboard instrument.

This is the most useful idea in the lecture, and it generalises well beyond synthesizers: the control surface a designer chooses determines the repertoire that becomes possible on the instrument, and therefore the music that gets made. Smith calls Buchla underappreciated, notes he designed for forty or fifty years, and adds, with evident affection, that Buchla probably preferred it that way.

The single-note problem

After the modular systems came the portable instruments: the Minimoog, the ARP Odyssey. These removed the patch cables, which made them friendlier, but they kept a problem that Smith clearly enjoyed describing. They had keyboards, so people walked up and played a major triad, and got one note. That is not how a keyboard is supposed to behave, and it confused people.

His defence of the monophonic instrument is that plenty of real instruments play one note at a time. Brass and woodwind do. The skill the early synthesizer asked for was shaping that single note in real time with knobs and controllers, and a generation of players learned to do exactly that. Smith bought a Minimoog in 1972, largely because it looked interesting, and found that it sat precisely at the intersection of his engineering and music backgrounds. Roughly 12,000 Minimoogs were built in the original production run. A number he offers to make the point that this was a small, non-mainstream market.

The Prophet-5: a microprocessor and a shelf of chips

Smith founded Sequential Circuits in 1974, initially building sequencers and programmers as accessories for other people’s synthesizers. Having worked with microprocessors in Silicon Valley, he saw what a new chipset from Solid State Music (SSM) made possible: with SSM oscillators, filters, envelope generators and VCAs on one side and a microprocessor on the other, a polyphonic synthesizer could be made fully programmable without great expense.

The Prophet-5 was designed in 1977 and shipped in 1978, announced, Smith notes, at a NAMM show small enough to fit in the basement of the Disneyland Hotel. It was the first fully programmable polyphonic synthesizer and the first musical instrument with an embedded microprocessor, and its significance is best understood through the complaint it answered. Now a player could hit a major triad and get three notes. Better, they could press a button and have a completely different sound. One keyboard could be strings, then brass, then a lead, then a bass, quickly and repeatably. Oberheim followed, then Roland’s Juno and Jupiter lines, and the idea spread fast.

Smith is careful not to overstate the scale. Around 7,000 Prophet-5s were sold, by his recollection. Published sources more often put the figure near 6,500 across all three revisions, of which Revision 3, which replaced the SSM chipset with Curtis CEM parts, accounts for the large majority. Either way, the instrument that defined an era sold in the thousands, not the hundreds of thousands. The operating system, he notes almost in passing, ran in about 4 KB.

From USI to MIDI, without a standards committee

The chain of reasoning Smith describes is short and worth following. Once an instrument contains a microprocessor, it is nearly trivial for it to talk to a microprocessor in another instrument. Several companies reached that conclusion at once and each built its own link: Sequential had a high-speed serial interface, Roland had DCB, Yamaha and Oberheim had their own. Which meant that the obvious remaining step was for an instrument from one company to talk to an instrument from another.

Smith presented that proposal at an AES convention in New York in 1981, under the name USI. Universal Synthesizer Interface. He is explicit that it was never meant to be the finished thing: it was a starting point, published to get companies into a room. Bob Moog wrote the emerging standard up in Keyboard in October 1982.

The part usually left out of the story is that the first meeting failed. Smith convened manufacturers at NAMM in January 1982. The discussion fell apart over cost and over competing technical wishes, including a faction wanting 16-bit parallel at high speed. He left it, in his own account, dejected and expecting nothing to happen. What saved it was that afterwards, engineers from Roland and other Japanese companies approached him privately and said they still wanted to do it. The work then proceeded through 1982 with Roland, Yamaha, Korg and Kawai. The Prophet-600 shipped that December as the first MIDI-equipped instrument, and at NAMM in January 1983 a Prophet-600 and a Roland Jupiter-6 were connected in public and worked.

Smith’s explanation for why it came together quickly is the single most quotable thing in the lecture, and, as transcribed, he says they were able to move fast because they conveniently skipped working with any sort of standards committees and just did it on our own. He is equally frank about the aftermath: “not invented here” resistance from companies that had stayed out, complaints from firms that felt unconsulted, and academic papers arguing the job had been done incorrectly. Within a few years the same academic community was using MIDI to drive experimental controllers, because it happened to be well suited to that.

He also identifies MIDI as the origin of the home studio. Computers of the period were far too slow to handle audio, but they could sequence a room full of keyboards and a drum machine, so the personal studio arrived as a MIDI phenomenon first, and only later, as machines got faster, did real audio move in alongside it.

The digital dark ages

Then, as Smith tells it, the Yamaha DX7 arrived in 1983 and the analogue instruments began to drain away. It was cheaper because it was digital, it had sixteen voices and velocity, and it sounded new. His own theory about why it took hold is deflationary: it sounded something like a Rhodes piano without weighing two hundred pounds.

The argument sharpens with the Korg M1, which arrived in 1988: Smith says “87, maybe”, and hedges it. The M1 was the first sample-based workstation, the first keyboard on which a player could get a convincingly real piano, and it became the best-selling synthesizer ever made. Smith’s reading is that this revealed what most musicians had actually wanted from a polyphonic analogue synthesizer all along: not synthesis, but the best emulations they could get. Strings, brass, piano, organ. Analogue instruments are bad at pianos. The M1 was good at them.

He calls the following two decades the digital dark ages, on the grounds that the industry spent them building progressively better M1s (more voices, higher sample rates, better samples) while synthesis as such largely stopped being interesting to the market. It is a partisan reading from a man who makes synthesizers, and he says so, acknowledging that if you play in a band you genuinely need convincing piano and organ sounds and that there will always be a market for them. But the framing is his, it is sharp, and it is why this lecture is worth reading rather than summarising.

Software, and why its own designer stopped playing it

In the mid-1990s Intel was promoting Native Signal Processing, an initiative whose purpose, Smith says drily, was to find ways of consuming the processing power Intel wanted to keep selling. Intel approached Seer Systems, where Smith was working, and the first result was a software synthesizer running on a 486 (assembly code, fixed point, a simple General MIDI instrument) which Andy Grove demonstrated in a COMDEX keynote. A Pentium-era rewrite in floating point followed and was sold to Creative Labs, who combined its 32 voices with 32 in hardware and advertised 64.

Seer’s professional software synthesizer, Reality, shipped in 1997 (Smith recalls “maybe 96”). It did subtractive synthesis, sampling, FM and some physical modelling, and Smith admits they billed it as the future of music synthesis, which he concedes was mostly right. His account of demonstrating it is the best passage in the lecture: they set up a PC and a MIDI keyboard at a trade show, people liked the sound, asked what it was, were told it was a software synth, and could not grasp where the instrument actually was. The idea that a computer was calculating a synthesizer in real time was simply foreign.

What follows is the lecture’s turn, and it is a designer’s admission rather than a technical argument. Smith realised he never played Reality. Asking himself why, he concluded that the experience was staring at a screen, clicking a mouse and dragging things, then turning round to a keyboard to play a few notes, then going back to the mouse. A musical instrument, in his account, is something you interact with, something with personality, and it is not supposed to feel like work. So he drifted away from software at precisely the moment the industry was moving into it.

Back to hardware, and the return of the modular

Helping Roger Linn on the AdrenaLinn effects unit reminded Smith how much he liked hardware: something you can hold, something you control completely, without living inside somebody else’s operating system. He founded Dave Smith Instruments in 2002 and began with the Evolver, a small analogue/digital hybrid desktop synthesizer. The same year, Bob Moog reintroduced the Minimoog as the Voyager.

Smith is good on how much easier the engineering had become. Designing in the 1970s was, in his telling, genuinely hard. By 2002 you could lay out a board with cheap tools, email it, and have it back in a week, and processors were fast enough that code no longer had to be assembly. He also wanted a smaller company than Sequential, which had reached about 170 people, too much work and not enough fun. He tried a one-person company for around five years before concluding that a few of the right employees are worth having.

Meanwhile two things were happening in the market. Players were hunting down old instruments. Prophet-5s, Oberheim OB-Xs, Juno-106s, and especially the discarded Roland TR-808 and TB-303, which had been written off commercially before Detroit and Chicago producers built entire genres on them. And Eurorack exploded, with a generation who had grown up on software buying small modules and patching them together. Smith notes the loop closing: those racks look a great deal like what Buchla and Moog were building in the 1960s.

His observation about the Prophet VS is the general rule. It sold poorly, was cleared out, and became sought-after a few years later. Musical instruments, he argues, break the rules that govern the rest of electronics, which is a large part of why he found them worth designing.

What Smith expected to last

Two predictions run through the closing section, and both have held up reasonably well in the decade since.

The first is that subtractive synthesis would endure, because after fifty years everyone knows what it sounds like. Turn a cutoff or resonance knob and you know what will happen. Smith contrasts this with FM and other methods that are harder to interact with, and with granular synthesis, which he rates highly as a sound but considers too narrow a palette to carry a standalone instrument, and therefore better suited to software. He had tried physical modelling in Reality and abandoned it on a logical point: if you had a perfect model of a violin, the best way to control it would be something you bow, and at that stage you have arrived back at a violin.

The second is about longevity, and it is the sharpest thing in the lecture. A Prophet-5 in 2016 cost more than it had when new. A DX7, despite selling in the hundreds of thousands, could be had for very little. There is, Smith says, no market for vintage digital, and then he asks the question that answers itself: can you have vintage software? Not really, because you would need a vintage machine to run it on. His conclusion is a commercial claim he could make with a straight face: buy this hardware instrument and it will still work in twenty or thirty years.

The related idea is his account of why analogue instruments have character. Analogue designers, in the transcribed phrasing, try their best to make every voice sound exactly the same, and digital designers try their best to make every voice sound different, and the analogue designers always fail slightly, which is the point. Every real acoustic instrument is a set of individual tone generators, one per string or key. Only in digital instruments is everything calculated from the same model. He connects this to the guitar market, where players will buy several nominally identical instruments and choose between them by feel, and asks why a synthesizer should be any different. It is also the root of his long-running irritation at the word workstation: an instrument should not be named after work.

There is an engineering coda. The Prophet-6, released in 2015 as a deliberate descendant of the Prophet-5, could not use the SSM chips that made the original cheap to build, because they are long gone. Surface-mount parts had become small and cheap enough that Sequential simply built the oscillators and filters from discrete circuitry instead, more components, lower cost. Underneath, high-speed DSP generates over a hundred control voltages at audio rates to keep the analogue circuitry in order. The instrument is analogue in the way that matters to players and thoroughly digital in the way that matters to the designer.

From the floor

The Q&A adds three things worth keeping.

Asked whether a good controller with knobs closes the gap to a hardware instrument, Smith’s objection is not latency but mapping: on a dedicated instrument the oscillator controls are where the oscillators are and everything is labelled, whereas a generic controller reused across several software packages leaves the player remembering that the third knob might be resonance. The one-to-one relationship with the engine is the thing being lost.

Asked why so few genuinely new instruments appear, he gives a chicken-and-egg answer about muscle memory. It is very hard to persuade anyone to invest years learning an instrument that may not exist in five, particularly when it comes from a small company. Keyboard skill transfers to anything with keys. Skill on a novel controller transfers nowhere. He points to serious efforts (Roger Linn’s work, the ROLI Seaboard, the Haken Continuum) while doubting the commercial odds, and an audience member counters with Harry Partch’s instruments and the Chapman Stick. Smith’s own nomination for the most successful new controller of the previous thirty years is the turntable, “like it or not”.

Asked where MIDI goes next, he notes that thirty-five years on it was still MIDI 1.0 and still doing what almost everyone needed. That it now runs over USB, so speed is not the constraint. And that SysEx always allowed anything anyone wanted. The limitation, he argues, was never the protocol but the absence of instruments that respond meaningfully to expressive controllers, and with twelve people at the company, he would rather spend the time on new instruments. MIDI 2.0 was ratified in 2020, four years after he said this, and addressed exactly the per-note expression gap the questioner was pointing at.

On artists, he declines to name anyone he disliked, says he will not take full blame for 1980s synth-pop, names Radiohead as a favourite and Pink Floyd’s The Wall among the records his instruments appear on, and says the real payoff of designing instruments is hearing someone else make music with them.