(Fr. son dessiné; Ger. graphische Tonerzeugung; It. suono disegnato).
Sound that is created (or controlled) from graphic markings made directly onto film, or onto paper or cards, photographic images of which are assembled on film; these markings are played back using a film projector and a conventional sound system. The various approaches to drawn sound have included the construction of special electronic instruments based on similar principles.
HUGH DAVIES
Films with integral ‘optical’ soundtracks were introduced, in several countries, between 1927 and 1929, after nearly ten years of development; the fact that the sounds of music and speech were reproduced directly from outlines that could be seen on the soundtrack inspired a number of film makers (mostly animators) and composers to explore the creative possibilities of ‘visible’ sound. The earliest experiments in drawn sound took place at about the same time as film makers, especially in the Soviet Union, were starting to investigate the possibilities of sound collage. In Leningrad in 1929 Arseny Mikhaylovich Avraamov and Yevgeny Sholpo drew directly onto film with a pin dipped in Indian ink. Finding this work too intricate (the width of an optical soundtrack is between 1·93 and 2·5 mm, depending on the system used), they turned to different methods, Avraamov working in Moscow and Sholpo in Leningrad. By photographing individual drawings Avraamov produced the first drawn film soundtracks in 1930 for the films Plan velikikh rabot (‘Plan of great works’) and Kem bït' (‘Who to be’), followed by Gibel' sensatsii (‘The end of a sensation’) in 1931; later he used the technique to recreate the microtonal inflections of the traditional music that he had collected in various parts of the Soviet Union. In 1932–3 he directed a special drawn sound laboratory in Moscow, the Laboratoriya Risovannogo Zvuka. Sholpo’s first independent project was to produce melodies assembled note by note from recordings of conventional instruments; he then explored a similar approach to that of Avraamov, which led him to develop his Variafon (a photoelectric composition machine) in 1932. Drawn sound was also used for part of the soundtrack of Romance sentimentale (1930), a short film made during a visit to Paris by two Russian film directors, Grigory Aleksandrov and Sergey Ėyzenshteyn [Eisenstein], neither of whom pursued the technique further. Other work carried out in the USSR in the early 1930s was done by the film animators Ivanov, Nikolay V. Voinov and Sazonov under the group name Ivvoston around 1933, and by N.Y. Zhelinsky; they were primarily concerned with realizing well-known pieces of music synthetically.
In Europe drawn sound experiments were made by the leading silent-film composer Edmund Meisel before his early death in 1929. Oskar Fischinger (1900–67), working at the Bauhaus in Dessau, experimented with hand-drawn sound in 1931 (only a demonstration film resulted); further work was done at the Bauhaus around 1932 by the composer Paul Arma. Perhaps as early as 1929 the Swiss Rudolf Pfenninger began work on similar lines in Munich, making a demonstration film Tönende Handschrift (‘Sonorous handwriting’) in 1932 and soundtracks for puppet and cartoon films soon afterwards. Also in 1932 Fischinger’s former colleague at the Bauhaus, László Moholy-Nagy (1895–1946), made Tönendes ABC (now lost) in Berlin, in which the shapes drawn on the soundtrack were also shown as the visual element of the film. The Australian musician and film editor Jack Ellitt, working in London, experimented with drawing directly on film from around 1932. In 1933 Arthur Honegger and Arthur Hoerée included drawn sound in their film score Rapt, and Hoerée subsequently continued this work, calling it ‘zaponage’ (retouching).
In the USA Ub Iwerks (1901–71), one of Walt Disney’s leading animators, used the medium for certain sound effects in a cartoon film, The Village Barber, which he made independently of Disney in 1930. The film director Rouben Mamoulian included a brief passage of drawn synthetic high and low frequencies in Dr Jekyll and Mr Hyde (1932). The best-known practitioner of drawn sound, the Canadian animator Norman McLaren, had his first drawn soundtrack – for his film Book Bargain in 1937 – turned down by his employers, the GPO Film Unit in London. In 1939–40 he worked in New York, where he made three short films with hand-drawn images and sound, and Rumba, which consisted of a soundtrack only. He then joined the National Film Board of Canada (NFBC) in Ottawa (from 1956 in Montreal), where he produced many award-winning films, some with drawn sound, including Synchromy (1971), in which the images are based on the patterns of the soundtrack. A single drawn sound of the notes of the overtone series, which produce a phantom fundamental, is featured in the score by Bernard Herrmann (1911–75) for The Devil and Daniel Webster (1941). In Hollywood from 1940 the brothers John and James Whitney carried out experimental work with a specially constructed system of 12 linked pendulums, which delineated waveforms that could be filmed; they used the system to create the soundtracks of their series of films, Five Film Exercises (1943–4).
After World War II McLaren was the major figure and influence in this area. Two NFBC staff composers who frequently collaborated with him, Maurice Blackburn (1914–88) and Eldon Rathburn (b 1916), each created drawn sound as part of a score for a McLaren film (Blackburn also used it in two other films), and a younger French-Canadian film maker at the NFBC, Pierre Hébert (b 1944), began to use the medium in 1966. The Italian film maker Cioni Carpi included hand-drawn sound in two short films which he made in Canada in 1960–61, and in Paris the composer Robert Cambier used it for several film scores from 1957. The Austrian film maker Kurt Kren used drawn sound in two films in the late 1950s, beginning with Versuch mit synthetischem Ton (1957).
Since the 1960s comparatively little use has been made of the technique, possibly because magnetic tape and the synthesizer offered increasingly flexible and sophisticated facilities. In Soundtrack (1970), by the American Barry Spinello, some of the images are identical with the hand-drawn soundtrack, and part of the soundtrack is created from fragments of patterned transparent self-adhesive plastic; during 1967–71 he also made Sonata for Pen, Brush and Ruler and Six Loop-Painting. Similar derivations of optical images from the soundtrack include Lis Rhodes’s Lightmusic (1974), Pierre Rovère’s Black and Light (1975) and Robert Russett’s Primary Stimulus (1979). A reversal of this process is the production of a soundtrack from visual images, treating them as if they were drawn sound; this possibility was explored in Tony Conrad’s Articulation of Boolean Algebra for Film Opticals (1974) and in two films made by Guy Sherwin in 1977, Musical Stairs and Speed and Sound, in which the soundtracks were created from the images of a flight of exterior metal steps and the changing pattern of railway lines filmed on a train journey.
The images that appear on an optical film soundtrack are created by photographing a beam of light that is modulated by sound-waves. The soundtrack is played back by reversing the process: sound-waves are produced by the variations in the amount of light passing through the track and striking a photoelectric cell in the film projector. The two basic methods of recording sound optically on film employ respectively an image of variable area or width, and one of variable density. In the first the sound appears as a continuous irregular line forming the boundary between the areas of dark emulsion and transparent film, or as a double-edged symmetrical section defined by two such lines in the centre of the dark strip. In the second, which is less frequently used, the degree of shade affects the amount of light that reaches the photoelectric cell.
The exponents of drawn sound experimented with both these methods of recording sound on film. The variable area technique was often applied using not a continuous line but isolated images: Moholy-Nagy, Fischinger and Avraamov used geometric shapes (Avraamov restricted himself to triangles only); Moholy-Nagy also experimented with letters of the alphabet, facial profiles and fingerprints, and Avraamov with facial profiles; and McLaren and Spinello explored the use of repeated patterns of closely drawn lines or dots (fig.1). A few people have drawn directly onto that section of the filmstock on which the soundtrack is recorded, but since the maximum width of the track is 2·5 mm this process was explored mainly by animators such as McLaren who also drew the images for some of their films directly onto the stock. The same idea was essayed briefly by Avraamov and Sholpo in 1929, by Ellitt around 1932 and by Spinello from 1967. In most cases drawings made on paper or cards are photographed, often individually, and the images transferred frame by frame (as with cartoon animation) onto the filmstock to produce continuous sounds; normally the whole width of 35 mm film is used for the first stage of this process. Some experimenters assembled a whole library of sound-wave patterns on individual cards, each of which produces a semitone step in a wide range of pitch, with a set of cards for each timbre and sometimes additional cards for microtonal steps and glissandos, as well as masks to determine the dynamic level. Voinov created a card system of this type which had a range of 87 notes. Working on the basis of a technique devised by Pfenninger, McLaren used cards on which a basic sound-wave pattern measuring 30·5 by 5 cm, was repeated between four and 128 times to produce a range of five octaves, a sixth being obtainable by filming at double speed; 24 degrees of volume were possible. Pfenninger’s card library had a similarly large pitch range and included speech waveforms.
Variable density techniques were used primarily in the USSR by the early experimenters with the Shorin system, and by McLaren in his New York films (1939–40). McLaren refined this system by adding elements of variable area techniques such as a repeated shaped outline at one edge, and patterns of closely drawn lines, lighter in shade than their surroundings.
Some of the variables in filming a drawn soundtrack include the distance of the camera from the image, the camera speed, the exposure time and the possibility of superimposing different layers by means of multiple exposures; as an alternative to multiple exposures, parallel layers may be recorded by subdividing the width of the soundtrack into two or more separate channels, or an intermediate mixing stage may be used. These technical procedures may affect pitch, volume and timbre, depending on whether the system is one of variable area or variable density.
A number of musical instruments have been invented in which drawn sound and related techniques either generate notes or control their characteristics, but none has been manufactured commercially. In some cases inventors devised special machines or techniques for drawing sound. The first three of Sholpo’s four Variafons, constructed between 1932 and 1946, still featured filmstock as the storage medium, while in Moscow in the mid-1930s (between assisting first Avraamov and later Sholpo) Boris A. Yankovsky developed a system of filming tone-wheels; a similar process was used in Ivan Eremeeff’s ‘universal recorder’, devised around 1934 for the preparation of the wide film strips used as pitch and timbre masks in his Syntronic organ. In the Whitney brothers’ pendulum system (early 1940s) the subsonic oscillations were filmed at a speed some 60 times slower than normal to produce the pitch range required. A related principle was used in the light-screen devised by Michel Waisvisz in the mid-1970s for use in exhibitions; a large ‘torch’ containing a photoelectric cell controls the frequency of an oscillator as it is moved (by hand) in front of a score consisting of several rows of soundtrack-like patterns. A more recent example of an application of film is in a mechanical musical instrument, the ‘flute-playing machine’ (1980) of Martin Riches, in which the pitches are drawn on a roll of film 15 cm wide.
Soundtrack-like masks (sometimes created by cut shapes of tape stuck onto clear film) were also used in devices developed for electronic music studios during the 1960s, for controlling complex functions such as dynamic envelopes and switching multiple oscillators or tape tracks on and off (as in a player piano roll). Examples include Hugh Le Caine’s multi-channel Spectogram controller (in the studios of the National Research Council, Ottawa, the University of Toronto and McGill University, Montreal), Myron Schaeffer’s Hamograph (University of Toronto), Fernando von Reichenbach’s ‘sound level photoprogrammer’ (Instituto Torcuato di Tella, Buenos Aires), the Photoformer (NHK Electronic Music Studio, Tokyo) and an optically-controlled effects generator at the radio studios in Prague and Plzeň.
A number of keyboard instruments were based on rotating photoelectric tone-wheels – celluloid or glass discs on which photographed or hand-drawn waveforms were recorded; these interrupted a beam of light and affected its reception by a photoelectric cell, functioning in the same way as in the film projector. The idea was pioneered in 1916 by the South African Hendrik Johannes van der Bijl (1887–1948) at Western Electric in New York. The Hugoniot organ, built in 1921, the Cellulophone (c1927), the Superpiano (1927), the Photona (c1933–5), the Polytone (c1933) and the Radio Organ of a Trillion Tones (c1930) employed the principle, as did the photoelectric siren invented by J.F. Schouten at the Philips research laboratories in Holland (c1938). Since World War II, with the increasing sophistication and reliability of electronic circuitry, such instruments have been rare. Since 1984 Jacques Dudon has created 400 graphically elaborate ‘photosonic’ discs, many of them computer-designed, for several specially constructed ‘lumiphones’.
Following Sholpo’s Variafon, several other composition machines were developed in which the sounds are ‘programmed’ or generated, or both, by photoelectric means. Transparent film 1·5 metres wide carries the notation in the fourth Cross-Grainger free music machine (1953–61). The Oramics system (1962–5; fig.2) has ten parallel tracks of 35 mm film, the full width of each of which is used for an individual aspect of the sound; this system was inspired by the idea of reversing the process of producing visual images from sound on the cathode ray tube (CRT) of an oscilloscope. The Composertron (c1948) is based on a CRT television screen on which sounds are drawn; it anticipated the use of screens in computer music, for example in conjunction with a light-pen (at the Bell Telephone Laboratories, Murray Hill, New Jersey, from around 1966, and, from 1979, as part of the Fairlight CMI system). Other composition machines that have used the principle of drawn sound are the ANS in Moscow (c1950–57), which developed aspects of Sholpo’s work by having the composer scratch notations on a large blackened glass plate, controlling four photoelectric tone-wheels, and the Hanert Electrical Orchestra (1944–5), in which a scanning device reads markings in electrically conductive graphite on a set of cards by direct electrical contact. The Bildabtaster, which was used with the Siemens Synthesizer from about 1960, scanned hand-drawn slides.
Film soundtrack also inspired two optical recording systems marketed in the 1930s, the Selenophone (1931) and the better-known, variable-width Philips-Miller system, devised by J.A. Miller at Flushing, New York, in 1931 and manufactured in a modified version by Philips at Eindhoven from 1936 until the late 1940s; these were superseded by the tape recorder. A related approach is that of instruments that use optical methods for recording sounds from other sources, which are often visually indistinguishable from hand-drawn sound. This was the basis of the ‘singing keyboard’ (c1936), in which short lengths of pre-recorded film soundtrack were triggered when the player depressed keys on the keyboard; today it would be called an analogue Sampler, like the more effective recent magnetic tape-based Chamberlin Rhythmate, Mellotron and Birotron. Two electronic organs based on photoelectric tone-wheels containing ‘sampled’ pipe organ recordings were the Hardy-Goldthwaite organ (c1929–30) and the Welte Lichtton-Orgel (c1933) (see Electronic instruments, §I, 3, fig.2c). The domestic Optigan Music Maker (marketed in 1971–3 by the toy manufacturer Mattel) and the Orchestron, invented by David Van Koevering (after purchasing the Optigan designs) and manufactured by Vako in 1975, were also analogue samplers; both used a single tone-wheel for the complete pitch range in each timbre, which on the Optigan consisted of removable 30 cm flexible discs containing both timbres and automatic accompaniments (laser technology permitted greater storage capacity).
Pre-recorded speech waveforms on photoelectric glass tone-wheels were used from the early 1930s to the 80s in the ‘speaking clock’ telephone information service in many countries, and photographically recorded sound waves were the basis of the Visible Speech machine developed at the Bell Laboratories in 1948 as an adjunct to the Vocoder. High-capacity digital photoelectric storage for both sound and computer systems is common today, in the form of the laser-based compact disc and optical computer disc.
Some instruments and sound installations involve human movement that affects a photoelectric cell in the manner of a mask, as in the Saraga Generator (c1931), Qubais Reed Ghazala’s recent similar Photon clarinet and his Video Octavox synthesizer, several installations by Christopher Janney (including Soundstair, 1977, and the recent Harmonic Runway) and Jacques Serrano’s Mur interactif spatio-temporel (1984) with 3072 photoelectric cells; infra-red beams are used in a similar manner in Donald Buchla’s MIDI controller Lightning (1991), and in Interactive Light’s Dimension Beam (1993), originally used in a video game, which was developed by Roland and incorporated as the ‘D Beam’ in several electronic keyboards from 1998. A similar approach treats visual images as if they were drawn sound, using video cameras whose images are analysed digitally, as in the DIMI synthesizer (1971) and performance systems such as the Oculus Ranae developed by Douglas Collinge and Stephen Parkenson (c1985), Fred Kolman’s Kolman Kube (1989), and Kristi Allik and Robert Mulder’s Pentaprism (1989).
Finally several systems have employed drawn sound without the photoelectric element. Drawing movements on a large ‘tablet’ (80×72 cm) are electromagnetically sensed in the UPIC system devised by a team led by Xenakis; although originally demonstrated in 1977, it did not achieve a real-time capability until 1987. In the Technos Acxel Resynthesizer (c1988) a finger can ‘draw’ waveforms and envelopes across a touch-panel of 32×64 squares, and a touch-screen is featured on the Wersi Pegasus synthesizer workstation (1993).
L. Moholy-Nagy: ‘Új filmkísérletek’ [New film experiments], Korunk, viii (1933), 231–7; repr. in K. Passuth: Moholy-Nagy (Budapest, 1982); Eng. trans. (London, 1985), 319–23
H. Rosen: ‘Synthetic Sound’, Wireless World, xxxii (1933), 101 only
Z. Pešánek: Kinetismus: kinetika ve výtvarnictví – barevná hudba [Kineticism: kinetics in art – colour music] (Prague, 1941) [illustrations of early drawn sound images]
L. Becker: ‘Synthetic Sound and Abstract Image’, Hollywood Quarterly, i (1945–6), 95–6
R.K. Potter: ‘Audivisual Music’, Hollywood Quarterly, iii (1947–8), 66–78
R.E. Lewis and N.McLaren: ‘Synthetic Sound on Film’, Journal of the Society of Motion Picture Engineers, l (1948), 233–47
W. Meyer-Eppler: Elektrische Klangerzeugung: elektronische Musik und synthetische Sprache (Bonn, 1949), 63–71, 111–14
N. McLaren: ‘Notes on Animated Sound’, Quarterly of Film, Radio and Television, vii (1952–3), 223–9
J. Whitney: ‘Bewegungsbilder und elektronische Musik’, Die Reihe, vii (1960), 62–72; Eng. trans., ‘Moving Pictures and Electronic Music’, Die Reihe, vii (1965), 61–71; excerpt from orig. Eng. version repr. in Experimental Animation: Origins of a New Art, ed. R. Russett and C. Starr (New York, 2/1988), 171–3
P. Ford: ‘History of Sound Recording IV: Motion Picture and Television Sound Recording’, Recorded Sound, no.12 (1963), 146–54
H.F. Olson: Music, Physics and Engineering (New York, 2/1967), 373–8
H. Davies: Répertoire international des musiques électroacoustiques/International Electronic Music Catalog (Cambridge, MA, 1968), 301–3 [first pubd as Electronic Music Review, nos.2–3 (1967)]
B. Spinello: ‘Notes on Soundtrack’, Source, no.7 (1970), 50–51; repr. in Experimental Animation: Origins of a New Art, ed. R. Russett and C. Starr (New York, 2/1988), 175–7
T.L. Rhea: The Evolution of Electronic Musical Instruments in the United States (diss., George Peabody College, 1972), 126–37, rev. as ‘Photoelectric Acoustic-Sound Instruments’, Contemporary Keyboard, iii/11 (1977), 62 only; repr. in The Art of Electronic Music, ed. T. Darter and G. Armbruster (New York, 1984), 14–15
J.H.M. Goddijn: Groot elektronisch orgelboek (Deventer, 1975), 220–24
R. Manvell and J.Huntley: The Technique of Film Music (London, 2/1975), 183–93 [incl. repr. of McLaren’s article, 1953]
R. Russett and C. Starr, eds.: Experimental Animation: an Illustrated Anthology (New York, 1976, rev. 2/1988 as Experimental Animation: Origins of a New Art, 116–28, 163–77
B. Schrader: Introduction to Electro-Acoustic Music (Englewood Cliffs, NJ, 1982), 70–74
R.S. James: ‘Avant-Garde Sound-on-Film Techniques and their Relationship to Electro-Acoustic Music’, MQ, lxxii (1986), 74–89
R.M. Prendergast: Film Music: a Neglected Art: a Critical Study of Music in Films (New York, 2/1992), 198–205 [partly paraphrases McLaren’s article, 1953]
J. Dudon: ‘The Photosonic Disk’, Experimental Musical Instruments, xiv/4 (1999), 36–46