Scroll Top

Inside the Sony lab making the world’s first AI music album


Artificial intelligence is getting creative and later this year Sony plan on releasing the first AI produced album. Will you buy it? and more importantly will you know the difference?


Some 70 years ago, computer scientist Alan Turing famously set the bar for Artificial Intelligence (AI) – to design a computer that could hold a conversation with a human and convince them it was a person.

However, on a spring afternoon in the Flow Machines laboratory, in Paris, senior researcher Pierre Roy was more concerned with his music making AI software’s ability to create a convincingly catchy song.


Watch: Sony released two new songs from an unsigned AI

“So far, from the technical standpoint, no one knows how to do a proper song, to tell a story,” he said, “it’s a hot topic in AI.”

Flow Machines, a project of Sony Computer Science Laboratories in Paris that receives funding from the European Research Council, is developing an AI program that can compose compelling, professional quality music, an aim that’s shared by similar ventures such as Jukedeck in the UK and Google’s Magenta project.

Ever since Turing defined his test, popular culture has fixated on the idea of sentient AI, both benign and catastrophically malign. But AI today has become something of a catch all term for software that augments human intelligence, usually by mining vast troves of data, and it’s become commonplace in everyday life, from asking Siri to find the nearest McDonalds to helping you discover new playlists on Spotify, and letting Facebook auto tag you or your friends in a photo.

What makes AI different from traditional data analytics programs is its capacity to teach itself to recognise patterns using machine learning algorithms, and then make autonomous decisions based on those patterns without being explicitly programmed to do so.


Companies are turning to synthetic data to fill in gaps in their AI models

Flow Machines’ technology is no different. The project’s AI machine is not a self-aware robot that yearns to write a love song – not yet, anyway. It’s a set of complex algorithms that has been given 15,000 songs to analyse.

“We don’t give the machine musical rules or abstract musical knowledge. It’s only the machine producing music based on what it learned from the data.”

When Benoît Carré, the project’s artistic director, wants to compose a new song, he sets his program loose on a selection of music from their database – a bossa nova playlist, perhaps, or his favourite Charlie Parker tunes. He runs an analytical model known as a Markov chain that identifies patterns in those selections and then imitates and varies them to create its own original composition.

The computer calculates the probability of certain chord progressions, melodic sequences, and rhythms, and uses these probabilities to generate new, plausible variations.


Nvidia's AI watched Pac-Man then re-created the game from scratch without coding

“We don’t give the machine musical rules or abstract musical knowledge,” explained Roy, “it’s only the machine producing music based on what it learned from the data.”

At this point in the process though the Flow Machines system requires the intervention of human expertise. Carré might keep an eight-bar phrase he likes and reject the rest, running the program again until he has a melody and a chord sequence that he’s happy with.

Last month, Flow Machines and Sony Music unveiled their first AI single “Daddy’s Car,” an upbeat pop song based on a selection of Beatles tunes. The song is tuneful, but undeniably a pastiche and somehow bewildering, triggering an odd nostalgia for a non-existent memory.


Daddy’s Car

It was accompanied by an irregular take on the Great American Songbook, the less accessible “Mr. Shadow.”


Mr. Shadow

All the stylistic flourishes – the harmonies, instrumentation, and the lyrics – are thanks to Carré. What the computer spits out is pretty basic. Songs are entered into the database in their most barebones form – a lead sheet with just chord labels and a melody – and that’s how the computer composes as well. Carré is responsible for all the production touches and details that give a song its distinctive sound.

This offers Carré a great deal of artistic license in the rendering and production phase, but also places constraints on what kind of music the program can interpret and produce.

“It only can be used for the type of music that can be represented by a lead sheet,” said Roy, “classical wouldn’t make sense… even techno or trance wouldn’t make sense. Only music based on harmonized melodies, that is Western popular music, that can be reduced to notes and chords.”


A new report backs up claims that AI could increasingly aid terorism


The limitations of this particular technology beg the question: When we thrill to a certain phrase of music, what is it that we’re responding to? And can we expect a machine to ever recreate an expression we consider to be so fundamentally human?

Marvin Minsky, who founded MIT’s AI Lab and is considered one of the forefathers of the field, notably hypothesized that humans take pleasure in self-contained, repetitive patterns and their variations because they allow us to comprehend and play with time.

Advances in neuroimaging have offered deeper insight into this idea. Neuroscience researchers from McGill University and the Rotman Research Institute have found that when people listen to music they enjoy, the parts of their brain linked to reward and motivation light up. Listening to passages of music that cause us to react with particular intensity – those that give us “chills” – initiates a rush of dopamine, the “reward” neurotransmitter associated with pleasure that has been linked to sex, food, and falling in love.

Interestingly, dopamine is released in anticipation of the peak emotional moment as well as during it.


Google launches BARD its ChatGPT rival and sees $170 Billion vanish


“We build expectations and delight when they are creatively violated, whether by composers or in the performer’s interpretations,” explained Jonathan Berger, a composer and professor of music at Stanford University.

By this logic, AI should, in theory, eventually be able to identify the patterns of expectation, disruption, deferral, and eventual satisfaction that we find the most pleasurable and produce melodies that give us chills – although “Daddy’s Car” suggests they have some way to go yet.

Even if the next song produced by Flow Machines appeals to a number of people, Berger is skeptical that the program is capable of producing truly great art – for the moment anyway.

Beyond pattern repetition and variation, he said, “music is multi-layered and unfolds in multiple time scales. Pattern replication generally lacks the large scale contexts that constitute a significant aspect of art.”

The Flow Machines team counter that this isn’t necessarily the point. They see their machine not as a way to side line artists but as a tool to augment human creativity, and hopefully send it off in novel directions that otherwise may not have been possible. Something we keep hearing time and time again…


Meta's AudioGen AI generates convincing sounds from just text


“What’s really nice is that the songs sound very different from anything else, including what Benoît was doing before,” Roy said of the material being produced by the lab.

Flow Machines will release a full album before the end of the year — the first ever composed by AI — and are inviting a range of artists to collaborate, including Ash Workman, British synth-pop band NZCA Lines, and electronic duo Darkstar.

Carré hopes to bring together different musical styles but link them under an over arching conceptual theme, taken from one of Hans Christian Anderson’s darker fairy tales about a writer who loses, and re-discovers, his shadow.

“It’s like a mirror, a play on identity, so it’s interesting with what we are doing,” he said, adding that there were also parallels to be drawn with his goal of AI contributing to stylistic diversity.

“The shadow, in the tale it is traveling, and the more it travels, the more it lives,” Carré said.

It bears mentioning that in the fairy tale the shadow eventually becomes corrupt, exchanges places with, and then kills the idealistic writer. So perhaps an echo of the malign AI is with us yet.


Researchers want to revolutionise AI by combining Quantum computers and Neural Networks


Ed Newton-Rex, a founder of Jukedeck in London, says his company, which offers to make custom soundtracks for videos, is also partly motivated by the idea of increasing access to what has historically been a somewhat rarefied industry — at least in the classical world in which Newton-Rex trained.

“I did nothing but music for 21 years, studied music at Cambridge, was going down route of becoming a classical composer,” he said. “In terms of democratisation, once computers can write music they can give tools to people who don’t have this huge musical education and let them get involved in the music writing process.”

But unlike Flow Machines, Jukedeck has begun experimenting with using AI for every step of the process, including synthesizing the computer’s composition into a fully realized track. The technology is still in its early days, but Newton-Rex sees no reason why computers couldn’t one day make the leap from pastiche to truly creative art.

“It’s hard to argue that even very disruptive innovations aren’t built on some preceding model and preceding experience,” he remarked, “having said that, part of the problem with asking if computers can be creative is defining what creativity is in the first place. We haven’t agreed as a society what it is and how it works.”


Survey shows images of deepfake faces are now more real than genuine photos


Jukedeck is using slightly different tech from Flow Machines: neural networks that imitate the human brain. These require much more training data than Markov Chains but, once trained, can be more powerful.

Roy and Carré freely admit that some of the implications and consequences of their creation may be out of their hands — and that’s part of the excitement.

“With rap, or disco, often it’s a new technology that creates a new style,” said Carré. He gestured to an Akai sampler sitting among an array of keyboards, headphones, and espresso cups. Launched in the 1980s, the Akai allowed artists in the emerging hip-hop scene to sample classic tracks and produce beats without a drum kit.

“At the beginning, a lot of people were afraid that the pianist and the drummers will be replaced, but it never happens this way,” Carré added, “it’s humans that find the ways to use technology to make interesting things.”

Related Posts

Comments (1)

Leave a comment


1000's of articles about the exponential future, 1000's of pages of insights, 1000's of videos, and 100's of exponential technologies: Get The Email from 311, your no-nonsense briefing on all the biggest stories in exponential technology and science.

You have Successfully Subscribed!

Pin It on Pinterest

Share This