One of the interesting things to come out of the Imagination In Action conference this past month was a pretty substantial interview with will.i.am (titled Harmonizing Creativity) conducted by our CSAIL director Daniela Rus.
will.i.am set the stage by introducing his experience with AI and related technology. He talked about how he met MIT professor Patrick Winston in the early days of the 21st century, and then started using AI for sound and video production, ultimately going on to get involved in the Beats company.
“I’m a musician because of the computer,” he said, describing coding and computer science as his early career aspirations, and talking about the intersection of tech and music.
In describing how AI has evolved, he talked about these early pioneers telling him that someday people would use language to talk to computers, instead of programming.
Those days, he noted, have now arrived.
will.i.am at MIT
“The concept of LLMs, we didn’t have that then,” he said. will.i.am described bringing something to his band in 2010 in a briefcase, showing them the potential of the nascent tech and saying “I’ve got the future right here.”
Lots of fans, he said, are familiar with the Black Eyed Peas lyric: “I’m so 3008, you’re so 2000 and late,” but that line, he added, was driven by the idea that we are embarking on this voyage toward innovation quickly.
“I’m a curious person,” he told Rus, explaining how he got in on AI on the ground floor. “I like asking questions.”
It occurred to him early on, he explained, that with the potential of machines, a lot of human activity would ultimately become obsolete.
“Why am I over here creating, when I can I can create something that creates?” he said.
will.i.am at MIT
will.i.am explained how he started investing in AI with companies like OpenAI and Anthropic, and
How he is now the CEO of FYI.AI. He keyed in a demo for the audience, starting with an AI responding to an avatar of Bob Marley. His AI respondent used the phrase “imagination in action,” which I thought was great (listen closely) but the quality of the conversation was amazing.
Here’s a part of the AI’s thoughts aimed at promoting diversity and democratizing tech:
“It's about showing every damn body that tech ain’t just for the select few. It's about putting voices like ours in the mix, making sure black women and peeps from the inner city can see themselves reflected in this tech revolution. Cha. It's about empowerment, showing that AI can speak our language, understand our struggles and amplify our stories. When folks see an AI (like this, it) bridges gaps, makes tech relatable and dismantles the notion that something as powerful as AI is out of reach for you. It's about inclusivity, breaking barriers and making damn sure that the future looks like every damn body.”
When asked how the company achieves this, will.i.am said FYI uses natural speaking voices instead of voice actors.
He also talked about how he runs a show with an AI named Fiona.
will.i.am at MIT
“People can engage with that costar every day,” he said, “one to a million, not that limited experience that we currently have with traditional media.”
Doing away with stilted conversation training, he suggested, gives you that real aspect that draws people in.
“They were blown away by how the AI feels when you communicate,” he said, describing an interview with De La Soul, when Rus asked how the guests respond to talking to an AI. “It’s a superstar. It is the star. It’s not just a tool.”
Going into more of what comprises FYI’s “secret sauce,” he suggested part of the technique involves “puppeteering” the model, but that natural language helps, too.
“You can’t (make the training data) like it was writing a book,” he said, contrasting FYI’s personas to traditional models trained more formally, which he characterized as “super stale” and “super bland” because of a lack of conversational authenticity. “The magic is how we gathered and recorded,
“We just had real conversations, with all the ‘ums’ and the ‘ahs’, instead of like, ‘perfect’ … and then two, the LLM has no clue that it’s speaking – it thinks it’s being read.”
With that said, he went into a lot more detail about how the LLM works with something he called TTS, a text-based component, and how to trail-blaze making those models vibrant.
Your imagination is doing all of the coloring when you’re reading text. … does a machine have imagination? No, it doesn’t. … so how do you get it to express? You’ve got to break the conventional text, and reading, and you have to start to think differently.”
Another interesting part of will.i.am’s interview was where he went through a survey of the history of music in the modern world, starting with the old days before things like recording and touring. I think this is interesting and important enough to showcase in some detail:
Of the late 1800s: “I'm a musician, I would play in some orchestra or some band, in theaters and classical music, there's no, I can't even imagine a music industry. There's no such thing as a music industry. There's no such thing as radio. There's no such thing as vinyl. There's no such thing as this current state of song structure. Songs were for telling stories (that) you composed for conductors to conduct an orchestra. So popular music is not today's concept of popular music.”
Of earlier ages before people had common recordings: “People listen to music at church. They listen to music at the theater, but do they all sing the same lyric, randomly, as they’re cleaning the house? That wasn't the experience. What did that? The recording industry did that. Every single industry when it comes to music, was because of some piece of technology. 1449, because of the printing press, shortly after that, you had your Mozart, your Bachs, your Beethovens, those folks have wrote composed because you can print sheet music for conductors to conduct and sell more oboes, … and all these different instruments to play the music that was printed in the sheet music, that was the first industry.”
Of the recording era, in the 1900s: “You have your rights, your mechanical rights, or contracts. A song is four minutes and some odd seconds. Anything over six minutes is part one and part two. Look at every jazz composition. Any song over six minutes is part two, because that's the amount of information that fit on lacquer. It’s a limitation.”
Then, bringing things up to date, he talked about the changes that AI is likely to usher in quickly.
Now we have new tech. So should this new tech be a part of recording tech? That would be idiotic. Why would you put abundance and infinite possibilities on an industry that's based on limitations. Why would you make AI make this type of music? A recording? Something that exists in a cage, a trap container, for three minutes?”
There’s a moment where he talks about a live musician or AI speaking directly to people in a crowd, another unprecedented use of tech in music. He explained some of the dynamics, vicarious aspects, of an audience that AI might have an impact on:
“We have an affection to love songs because it's a proximity to our love. It's not our actual love. We love heartbreak songs because it's a proximity to heartbreak. It's not your heartbreak. It's my heartbreak. I just shared my heartbreak with you, and you're like, ‘Oh yeah, it kind of feels like my situation.’ That's not your situation. We like party music, …. now you go to the club, and when you're in the club, the songs are like, ‘we're in the club.’ It’s telling you what you're doing. Like, ‘yeah, we are at the club. We're about to drink. Oh my gosh, this song is so on point,’ but it's proximity. It's not your actual (experience).”
Now, he said, we can make music more personalized, less vicarious, more direct.
“Let's make this thing do something new,” he said. “Let's make it make your actual heartbreak. Let's make it alive. What's your song?”
However, he said, it’s critical that people own their own data.
“There’s this broken practice,” he added, “we gave all of our data to these data monarchies.”
He also made a good analogy of people’s data to their nervous systems, or digestive systems. You can’t trade those other things around, he noted – so why data?
The challenge to humans, he added, is widespread.
“It’s not just Frank Sinatra,” he said, describing how AI can be trained on human endeavors, and what that might mean for the average person. “It’s your next door neighbor. … If you can simulate a building to not fall down in an earthquake, then you can simulate how somebody sells, how somebody loves, how nurses take care of someone, how people rap, all of their cadences…people need to own their stuff.”
In that vein, he also discussed his ongoing efforts to prepare new generations with a foundation started in 2008, College Track, that started with 65 students, but grew to 13,000 students headed for a variety of schools.
“Our students always aim to come to MIT,” he said. “It’s hard to get into, but our kids are, every single year, aiming to do that, especially the kids that are focused on computer science. … MIT is a special place, not only for me and my interest in AI, what sparked my interest in AI, but my passion and our purpose. … Our program that we tweaked is one of the most successful programs with how many schools that we have adopted in a school district, so 400 robotics teams serving over 13,000 students. Now we just want to continue to grow all that, to prepare the youth for tomorrow.”
Speaking of opportunity and challenge, Will.i.am acknowledged the point that we are at with AI. “It’s a new renaissance,” he said. “Humanity has these waves … this one’s an important one. It’ll change how we solve diseases, climate…I remain optimistic, I remain inspired by the engineers and the folks who have been working in this field before it was cool, before the hype, people who were building LLMs in the shadows, I salute those folks.”
His call for kind and compassionate results was inspirational:
“There are a lot of companies that are trying to get into AGI; what type of AGI are they getting into? Is it empathetic? Is it kind? Or is it analytical and judging?...I want to get to that, something that’s empathetic, that gives you the notion that you’re loved, that it cares … there’s a lot of lonely people out there. You have a lot of people who are afraid of the field out there. And when you get this job displacement, there’s going to be a lot of people that need to feel loved. … It’s imagination in action.”
