AI Helps Musicians and Scientists Finish Beethoven’s 10th Symphony

A public conversation about artificial intelligence has centered around whether AI can become sentient and thus the risk of robot overlords. While some have stoked such fears, many technologists who make or deploy AI want the world to know that it’s not about HAL from 2001: A Space Odyssey or Skynet from Terminator. They’re showing how AI is an effective software tool and can benefit humankind.

Chess Grandmaster Garry Kasparov once remarked that his loss to Big Blue did not teach us much about getting a computer to do what the brain does. To beat him, it took a team of computer scientists, chess experts, a vast library of games, and a machine that could process 200 million moves per second, which is not how humans play.

“If AI is defined as what humans do, then there is no artificial intelligence in the world,” said Wendy Pfeiffer, CIO at Nutanix.

As a user of AI tools in the data center, Pfeiffer leads an IT team that is leaving mundane tasks to machine learning – which she prefers to call it rather than AI – so that people can be more effective.

This was the motivation behind Professor Ahmed Elgammal’s project at Rutgers University to finish Beethoven’s 10th symphony. The composer’s 9th Symphony is considered his crowning achievement in the form. But afterward, Beethoven marshaled on, despite being deaf and having failing health, jotting down a few ideas for a 10th Symphony.

This was the challenge for Elgammal’s AI, to create roughly 10-minute movements from just a few fragments.

“Can we teach AI to learn from classical composers how to take a motif like that and develop it to a whole segment in a movement…and then with the second motif develop the second segment?” he posited during an interview with The Forecast by Nutanix.

In this Tech Barometer podcast, learn why Elgammal believes that while some music critics have denounced the idea of a computer writing music in Beethoven’s style, they misunderstand what its aims are.

“The goal is really to push the AI and the machine creativity forward so it can really become a creative collaborator to artists,” he said.

The idea that AI will replace humans is not remotely within Elgammal’s thinking. He sees the project as a learning exercise: AI assisting Beethoven in finishing music. His company, Playform, essentially does this. It provides a platform for artists to build their own AI to assist them in the creative process.

These kinds of AI tools are changing how people work whether in the arts, engineering, education, agriculture, transportation, medicine or in the data center. And its largely due to the cloud.

“There would be no progress in AI if there were no cloud,” said Holger Mueller, VP and Principal Analyst for Constellation Research covering next-generation apps and human capital management. “There would be no cheap compute and massive scale to do it.”

In this podcast, we explore an example of how AI tools are helping people to better themselves.

Transcript (unedited):

Jason Lopez: This is a story about artificial intelligence. It’s also a story about genius, about humanity, and about changing the world. This is the Tech Barometer podcast, I’m Jason Lopez. What we want to do in this podcast is show you what's going on in the field of AI. and it’s not a story with a dystopian movie narrative. The cloud is setting off a tectonic shift. It’s a platform that can help you learn to speak Icelandic, or land an airplane, or build a data center. Yes it's true AI can be kind of buzzwordy. But as the cloud matures, machine learning tools are poised to revolutionize the way people work and play. We're starting to experience natural language processing, such as Siri or Alexa or when you call into a bank for services and reach an automated attendant And there’s computer vision, which is what driverless cars use; or deep learning, which can analyze complex data like video or audio. An example of this is how Ahmed Elgammal computer science professor at Rutgers University helped to finish movements for Beethoven's 10th Symphony using AI. 

Ahmed Elgammal: Sketches of Beethoven that have around for a long time. There have been previous attempts to complete his symphony. 

Jason Lopez: Beethoven died before writing anything substantial that we'd call the 10th symphony. He left many musical fragments in the form of scores as well as notes scribbled on various scraps of paper. In the 1980s, musicologist Barry Cooper used a collection of Beethoven's sketches intended for a tenth symphony to put together a hypothetical first movement. But with a few new musical sketches discovered after Cooper's project, Elgammal's team had an intriguing challenge: essentially teach the AI Beethoven’s creative process, give the AI with just a handful of musical notes Beethoven scribbled down, and finish the 10th symphony.  

Ahmed Elgammal: The question is really how can we iterate and generate a symphony using AI because these kinds of sketches were very sparse to the point that any attempt to complete that same point has to be mainly inter interpolation of a human composer. So the question was, can I do something here? Is AI ready? 

Jason Lopez: The idea of a computer writing music in the style of a composer is not new. Twenty-five years ago David Cope, a University of California Santa Cruz music professor, who's now retired, produced music with a program called experiments in musical intelligence also known EMI, which he had developed since the early 1980s. Among the composers, EMI was learning from? Bach. 

David Cope: The reason I chose the inventions as sort of an initial output was because most of us who've taken piano lessons as children ended up struggling with these little contrapuntal nightmares because they are difficult, but they pose wonderfully interesting physical challenges and teach certain techniques that are indispensable, that no other pieces seem to do in quite the same way. What I want the project to do is to teach me about musical style and hopefully produce some good music that I like to listen to. Because I think that's a very, very important quality. 

Jason Lopez: Cope developed the program to the point of releasing albums of computer-generated music. The originality of EMI’s output depends on what you program into it. 

David Cope: You need a minimum of two works of the type you want to emulate, the more works, the more unique or individual the output will be. And the fewer works. The more like one of the works you input, the results will be. So therefore if you put two works in, there's a good chance that you will recognize pretty, you know, pretty clearly, some of the musical ideas relevant to one of the works in the database. If you put in a hundred works, the chances are remote that you'll be able to draw immediate connections by ear, at least to any one of the works that appear in the database. 

Jason Lopez: By the early 2000s, Cope had developed EMI to such a point that he renamed the program Emily Howell and released several albums of music under her name. At the end of this podcast, we’ll play music by EMI as well as by Bach and see if you can tell the difference. 

Ahmed Elgammal: Can we teach AI to learn from classical composers learning how to take a motif like that and develop it into a whole segment in a movement? And then with the second motif develop the second segment. 

Jason Lopez: Aside from being a computer science professor, Elgammal is a developer of an AI program called playform. It's a collaboration tool for artists. It makes AI programming accessible by allowing creators to build and train their own AI. His reason for completing Beethoven's 10th Symphony goes beyond being a classical music lover. It was to advance the capabilities of artificial intelligence. 

Ahmed Elgammal: There's a lot of things that need to be done in order to be able to take scores that are very fragmented and make a whole movement or two movements. To write minutes of music out of that, it's a big challenge, a lot has to really be done. 

Jason Lopez: That to-do list was a bit daunting. They had to feed AI Beethoven’s styles of writing forms like scherzos or trios. The way he develops melodic lines and harmonizes them. Counterpoint. His use of repeats, bridges and codas. And then there’s how Beethoven likes to divide the parts up among instruments in orchestrations. To take scant material, just a handful of fragments, required far more than filling in the blanks. It meant AI had to learn what would Beethoven do. 

Ahmed Elgammal: So, Beethoven just left small phrases or motifs. Exactly, like when you hear the 5th symphony and there are these four notes beginning the symphony. These four notes Beethoven would take and develop a whole movement. 

Jason Lopez: AI finishing Beethoven's 10th demonstrates something very powerful. This thing was a process. Because the process. It was not a matter of feeding a few scraps of music into a computer, hitting a button, and out pops a symphony. The reality is that it was a process. The first results from the AI didn’t sound like what Beethoven would do. It meant working with machine learning to develop the piece.  

David Cope: Computers were made by human beings… …not so mysterious. 

Jason Lopez: Remember the scene, the dawn of humanity, in 2001: A Space Odyssey? The primates discover tools for the first time… bones lying on the ground which they pick up as weapons. One of the apes, victorious, throws it into the air. As it descends back to earth the image jump cuts millions of years into the future, replaced by a spaceship headed for a space station. A none-too-subtle message that a jawbone and a space station are basically the same thing. Add to that, AI. In computer speak, AI thinks. But that’s jargon. It no more does actual thought than books actually remember information. Many technologists would say, if intelligence is defined as what humans do, then… 

Wendy Pfeiffer: There is no artificial intelligence in the world. 

Jason Lopez: Wendy Pfeiffer is the CIO of Nutanix, who points out the real meaning of all of this is that we’re creating more and more powerful tools which, based on algorithms, can make choices. 

Wendy Pfeiffer: But the machine is not doing anything beyond its programming. 

Jason Lopez: Computer science thinker and writer Mark Mills, puts into perspective: the cloud is transforming the world, with its constellation of information technologies from networking to IoT. Artificial intelligence is one of the critical tools the cloud enables. 

Mark Mills: They're giving individuals in real-time useful advice, which has extraordinary economic value because it's profoundly productive because it saves the most precious commodity in the universe: human time. 

Jason Lopez: Pfeiffer talks about the example of a machine learning tool from Move Works AI which enables employees to ask for help in their native language. It translates queries into machine-readable language. And executes on requests automatically. 

Wendy Pfeiffer: So even if I don’t have a help desk person who’s a native speaker of French… …workflows. 

Jason Lopez: A Deloitte study from 2019 notes the cloud is the driver of more than 80 percent of AI implementations in datacenters, as well as the platform from which users will access AI tools. And while some of the tools are enterprise level, helping IT people do their jobs in data centers or finding tumors on x-rays; a myriad of AI apps are coming to help us do better just about anything you can think of, such as finishing a work of music. Elgammal emphasizes this doesn’t mean AI is pushing humans out of the way – that wasn’t the reason for finishing Beethoven’s 10th. 

Ahmed Elgammal: Some people feel that creativity is something sacred to humans and, uh, machines should not step into that. Uh, I don't agree with that. If you understand the context of these kinds of projects, it's not about the machine will now make, be ton music or bark music or Mo music. Yeah. It can, and we prove it's possible, but that's not the goal. The goal is really to push the AI and the machine creativity forward. So it can really become a creative collaborator to artists. 

Jason Lopez: That collaboration seems limitless whether it’s helping people to compose music, design buildings or automate data centers. Elgammal’s quest is to understand how to make machine learning better. 

Ahmed Elgammal: I can't imagine a machine being called intelligent without having ability to be also creative to some degree. We all, uh, talk about AI in the context context of automation or maybe game playing, playing chess and playing, um, AlphaGo or driving cars, autonomously, uh, but, um, all that are important. Uh, but I think a fundamental thing is, is really AI becoming creative, um, and collaborative, creative, uh, uh, uh, partner to humans. 

Jason Lopez: Ahmed Elgammal is a Professor of Computer Science and the Director of the Art & AI Lab at Rutgers University, he’s also the founder of Artrendex, the company that makes Playform. 

You can create art with AI at playform.io. David Cope is an author, composer, and scientist. He’s a retired professor of music at the University of California, Santa Cruz. If you want to immerse yourself in a wealth of information, music and writing about algorithms, Cope’s webpage is artsites.ucsc.edu/faculty/cope. Wendy Pfeiffer is the CIO of Nutanix. Aside from her talk satirically entitled “Our Robot Overlords and the Future of Tech,” which you can find on YouTube, her latest project as CIO is the hybridization of IT work, which includes remote and asynchronous staff collaboration. AI tools are part of that effort. 

We also heard from Mark Mills, a senior fellow at the Manhattan Institute and a faculty fellow at Northwestern university's McCormick school of engineering. is author of the Cloud Revolution, How the Convergence of New Technologies Will Unleash the Next Economic Boom and a Roaring 2020s. 

Tech Barometer is produced by The Forecast. For more stories on technology check out theforecastbynutanix.com. You might be interested in Michael Brenner’s story, The Role of AI in Cloud Computing.  So, see if you can tell which of these compositions was composed by Bach in which was composed by David Cope’s Emmy. The first piece was Bach’s Invention 11 in G Minor, BWV 782. The second one was an invention by EMI. I'm Jason Lopez. This is the tech barometer podcast.

Jason Lopez is executive producer of Tech Barometer, the podcast outlet for The Forecast. He’s the founder of Connected Social Media. Previously, he was executive producer at PodTech and a reporter at NPR.

© 2022 Nutanix, Inc. All rights reserved.  For additional legal information, please go here.