In recent years, applications of artificial intelligence in creative fields have exploded. A new generation of image and text generators deliver impressive results. Now AI has also found application in music.
Last week, a group of researchers at Google Approved MusicLM – an AI-based music generator that can convert text announcements into audio segments. It’s another example of the rapid pace of innovation in the incredible years of creative AI.
As the music industry is still adjusting to the disruptions caused by the internet and streaming services, there is a lot of interest in how AI could transform the way we create and experience music.
Automation of music creation
A suite of AI tools now allows users to automatically generate music sequences or audio segments. Many are free and open source, such as Google’s Magenta Toolkit.
Two of the most popular approaches to AI music generation are: 1. continuation, where the AI continues a sequence of notes or waveform data, and 2. harmonization, or accompaniment, where the AI generates something to complement the input, e.g. B. Chords to walk with a melody.
Similar to text and image-generating AI, music AI systems can be trained on a number of different data sets. For example, you could augment a Chopin melody with a system trained in the style of Bon Jovi – as beautifully demonstrated in OpenAI’s MuseNet.
Such tools can be a great inspiration for artists with “blank page syndrome”, even if the artist himself is the final nudge. Creative stimulation is one of the most immediate applications of creative AI tools today.
But where these tools could one day be even more useful is in expanding musical expertise. Many people can write a melody, but fewer know how to skillfully manipulate chords to evoke emotion or how to write music in a range of styles.
Although music AI tools still have a long way to go to reliably do the work of talented musicians, a handful of companies are developing AI platforms for music creation.
Boomy goes the minimalist way: Users with no musical experience can create a song with just a few clicks and then rearrange it. Aiva takes a similar approach but allows for finer control; Artists can edit the generated music note by note in a custom editor.
However, there is a catch. Machine learning techniques are notoriously difficult to control, and generating music with AI is currently a godsend. You might occasionally strike gold using these tools, but you might not know why.
A constant challenge for the developers of these AI tools is to enable more precise and conscious control over what the generative algorithms produce.
New ways to manipulate style and sound Music AI tools also allow users to transform a music sequence or audio segment. For example, Google Magenta’s Differentiable Digital Signal Processing library technology performs timbre mapping.
Timbre is the technical term for the texture of the sound – the difference between a car engine and a whistle. The timbre transfer can be used to change the timbre of an audio segment.
Such tools are a great example of how AI can help musicians compose rich orchestrations and achieve entirely new sounds.
At the first AI Song Contest, held in 2020, Sydney-based music studio Uncanny Valley (who I work with) used timbre mapping to bring singing koalas into the mix. Timbre transfer has joined a long history of synthesis techniques becoming instruments in their own right.
Taking Music Apart Music creation and transformation are only part of the equation. A long-standing problem in audio work is that of “source separation”. This means you can split an audio recording of a track into its individual instruments.
While it’s not perfect, AI-powered source separation has come a long way. Using it will probably be a big deal for artists; Some of them will not like that others can “crack” their compositions.
Meanwhile, DJs and mashup artists get unprecedented control over how they mix and remix tracks. Source separation start-up Audioshake claims that this will open up new revenue streams for artists, allowing their music to be more easily adapted, for example for TV and film.
Artists have to accept that Pandora’s box has been opened, as was the case when synthesizers and drum machines first appeared, potentially replacing the need for musicians in certain contexts.
But pay attention to this area, because copyright laws protect artists from unauthorized tampering with their work. This is likely to become another gray area in the music industry, and regulation might struggle to keep up.
New music experiences The popularity of playlists has shown how much we like listening to music that has a “functional” use, e.g. B. to concentrate, relax, fall asleep or exercise.
The start-up Endel has made AI-supported functional music its business model and creates infinite streams to maximize certain cognitive states.
Endel’s music can be linked to physiological data such as a listener’s heart rate. Drawing heavily on mindfulness practices, his manifesto boldly suggests that we can “use new technologies to help our bodies and brains adapt to the new world,” with its frenetic and anxiety-provoking pace.
Other start-ups are also dealing with functional music. Aimi examines how individual electronic music producers can transform their music into infinite and interactive streams.
Aimi’s listener app invites fans to manipulate the system’s generative parameters like “intensity” or “texture” or decide when a drop occurs. The listener engages with the music instead of listening passively.
It’s hard to say how much heavy lifting AI can do in these applications — possibly little. Nonetheless, such advances guide companies’ visions of how the music experience might evolve in the future.
The future of music The above initiatives are at odds with several long-established conventions, laws and cultural values related to how we make and share music.
Will copyright laws be tightened to ensure companies that train AI systems on artists’ work compensate those artists? And what would this compensation be for? Do new source segregation rules apply? Will musicians who use AI spend less time making music or making more music than ever before? If one thing is certain, it is change.
As a new generation of musicians grow up with the creative possibilities of AI, they will find new ways to work with these tools.
Such turbulence is nothing new in the history of music technology, and neither powerful technology nor established conventions should dictate our creative future.
