Nvidia claims a new AI audio generator can make sounds never heard before

Nvidia’s New AI Music Editor Can Create “Sounds Never Heard Before”

Nvidia has announced a new AI music editor called Fugatto, which is capable of generating music, sounds, and speech using text and audio inputs it’s never been trained on. This tool allows for the creation of unique sounds, such as a trumpet that meows, by putting together songs based on wild prompts.

Examples of Fugatto’s Capabilities

As demonstrated in a video, Fugatto can produce music based on prompts like “Create a saxophone howling, barking then electronic music with dogs barking.” Other examples include producing unique sound effects based on a description, such as “Deep, rumbling bass pulses paired with intermittent, high-pitched digital chirps, like the sound of a massive sentient machine waking up.”

Editing Music with Fugatto

Fugatto can also edit music by isolating vocals in a song, adding instruments, and even changing up a melody by swapping out a piano for an opera singer. This tool can transform the sound of someone’s voice, changing their accent or giving them a different tone, like angry or calm.

Training Data for Fugatto

A paper released with the announcement shows the long list of datasets Nvidia says Fugatto was trained on, including a library of sound effects from the BBC. To build Fugatto, Nvidia says researchers had to put together a dataset with millions of audio samples. They then created instructions “that considerably expanded the range of tasks the model could perform, while achieving more accurate performance and enabling new tasks without requiring additional data.”

Availability of Fugatto

Nvidia does not say when, or if, the tool will be widely available.

Conclusion

Fugatto has the potential to revolutionize the music industry by allowing for the creation of unique sounds and music based on text and audio inputs. Its capabilities include producing music, sounds, and speech, as well as editing music and transforming the sound of someone’s voice. While the availability of the tool is unknown, Fugatto has the potential to change the way we think about music and sound production.

FAQs

Q: What is Fugatto?

A: Fugatto is a new AI music editor developed by Nvidia that can generate music, sounds, and speech using text and audio inputs it’s never been trained on.

Q: What kind of sounds can Fugatto produce?

A: Fugatto can produce unique sounds, such as a trumpet that meows, and can even create songs based on wild prompts, like “Create a saxophone howling, barking then electronic music with dogs barking.”

Q: Can Fugatto edit music?

A: Yes, Fugatto can edit music by isolating vocals in a song, adding instruments, and even changing up a melody by swapping out a piano for an opera singer.

Q: When will Fugatto be available?

A: Nvidia does not say when, or if, the tool will be widely available.

Q: How was Fugatto trained?

A: Fugatto was trained on a dataset with millions of audio samples, and researchers created instructions that expanded the range of tasks the model could perform while achieving more accurate performance.