Have you ever wondered if AI can mix and master music? Maybe you're seeking answers to topics: the best AI mastering, is there AI for music production, is there an AI bot for music, can AI make music, or maybe even can AI make my song sound better? Then check out EP 137 of the Inside The Mix podcast.
Unlock the secrets of blending traditional music mastery with the innovative force of AI as I engage with the Grammy-nominated wizard of sound, Jonathan Wyner. This episode is an audio trove for the sonically curious, as we dissect the mechanics behind AI's role in music production. You'll be privy to Jonathan's insights on the symbiotic relationship between AI and the human touch, deciphering the fine line where technology meets artistry.
Feel the pulse of music evolution with our deep dive into the world of mastering technology and AI-driven mixing services. The conversation takes an unexpected turn into the realm of audio element separation, where the once-impenetrable harmonies now lay bare before us, thanks to AI's ever-advancing reach. We get candid about the consequences of this accessibility boom, from the opacity of an AI's decision-making process to the brave new world where automated mixing platforms are already a reality.
As we wrap up, we address the elephant in the room: the nuanced challenges AI encounters in the mastering labyrinth, from grasping an artist's unique vision to the dynamic soul of music itself. We open the floor to a discussion on AI bias in music tech, setting the stage for the forthcoming AES Symposium on AI and the Musician in Boston. For anyone navigating the crossroads of DIY versus professional mastering, this episode is a beacon, illuminating the path to finding your sound in the age of AI.
Grab your FREETest Master at Synth Music Mastering TODAY! ✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸ Are you READY to enhance your music with my steadfast dedication to quality and personal touch? Bag your FREETest Master at Synth Music Mastering: https://www.synthmusicmastering.com/mastering
Buy me a COFFEE ✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸✸ If you like what I do, buy me a coffee so I can create more amazing content for you: https://www.buymeacoffee.com/marcjmatthews
Send a DM through IG @insidethemicpodcast Email me at marc@synthmusicmastering.com
Thanks for listening & happy producing!
Ian Stewart:
Hey, inside the Mix podcast fans, it's Ian Stewart. If you want to follow me or find out more info about me, the best place to do that is my website flotownmasteringcom. That's F-L-O-T-O-W-N. Masteringcom. You're listening to the Inside the Mix podcast. Here's your host, mark Matthews.
Marc Matthews:
Hello and welcome to the Inside the Mix podcast. I'm Mark Matthews, your host, musician, producer and mix and mastering engineer. You've come to the right place if you want to know more about your favourite synth music artists, music engineering and production, songwriting and the music industry. I've been writing, producing, mixing and mastering music for over 15 years and I want to share what I've learnt with you. Hello, folks, and welcome to the Inside the Mix podcast. If you are a new listener, a big, big welcome and make sure you hit, follow on your podcast player of choice. And to the returning listeners, as always, a huge welcome back.
Marc Matthews:
So I've just returned from an amazing mini break in Edinburgh with my fiance. She's been before, but it was my first trip to Scotland and, wow, what a city definitely up there in my top three cities I've ever visited. So we did all the touristy stuff. We walked up Arthur's Sea. It was pretty bad weather when we got there. I couldn't see over the edge, but I thought that added to the spectacle, as it were. We did get to see the view around Edinburgh on the third day when the weather cleared up and again, amazing stuff Tried some whiskey, of course, and then we went on a uh, an excursion for a day. So we were taken in a coach with a load of other people we didn't know and we were taken to Inverness and got to go on a boat on Loch Ness and then drive back down through Glencoe, which, wow, what scenery. That is, um, definitely up there, if not the best scenery I've seen in the UK and I am biased because I live in the southwest but wow. So if you're ever in Scotland, I highly recommend that drive through Glencoe. The scenery is second to none. Wow, amazing, amazing stuff. So that's enough about my recent excursion to Edinburgh.
Marc Matthews:
In this episode it's an interview episode and I'm joined by none other than Jonathan Weiner. Now, if you're not familiar with Jonathan Weiner, it's an interview episode and I'm joined by none other than Jonathan Weiner. Now, if you're not familiar with Jonathan Weiner, he's a Grammy-nominated mastering engineer and educator as well. We go into more detail in this episode with regards to that, but he's also the host, the face, the educator in the iZotope. Are you Listening series on YouTube, which is a fantastic series that I highly encourage you to go and check out, as I use it and reference it a lot, both in the podcast and when I'm working with clients with mixing and mastering.
Marc Matthews:
So in this episode, jonathan talks about the intersection of mastering and AI and how mastering is assisting music production. Jonathan talks about some common misconceptions about AI and mastering that he often encounters and, importantly, jonathan talks about what AI can and cannot do, both in mastering and mixing and in music production in general. Jonathan talks about what producers, artists and musicians should keep in mind when incorporating AI into their process and, importantly, jonathan gives advice to artists who are navigating the landscape of DIY mastering versus professional mastering services and what you should consider. So before we dive into this episode, I just want to make you aware of my 12 steps to a mastering ready mix checklist. It's a totally free checklist and with these 12 steps you'll be able to make the mastering process super smooth and exciting and make sure you can take your music up a notch in the mastering process. So head over to synthmusicmasteringcom, forward slash free and you can download that free checklist today. So that's enough for me, folks.
Marc Matthews:
Here's my conversation with Jonathan Weiner. Hey, folks, in this episode I am very, very excited now I say that every, but I genuinely am every time excited, but in particular this one to be joined by Grammy-nominated Mastering and Chief Mastering Engineer at iZotope, jonathan Weiner. Jonathan, thank you for joining me today, and how are you?
Jonathan Wyner:
I'm fine, I have to amend your introduction. I am in fact the Chief Mastering Engineer of MWorks Mastering. Also, I teach music production and engineering at Berklee College of Music. I've got a few other titles, but I am formerly the education director at iZotope, involved in a fair bit of product development and also creating some learning tools and social media and public speaking and all of that. But just to set the record straight, if you want to pretend that this was 18 months ago, then your introduction would have been entirely accurate.
Marc Matthews:
That will teach me. I thought I'd done my due diligence with my research there, but I was slightly out with that one there. So thank you for setting the record straight and I'm sure the audience will appreciate that. And so I've got your bio here, so hopefully I've got a bit of this correct. So I mentioned then a Grammy nominated mastering engineer, producer, educator and musician, and you lead the development of groundbreaking audio processing technologies, as you've mentioned, and you also teach at Berkeley College of Music and where you teach mastering and audio production. So you've got over three decades of experience in the industry and you've worked with a diverse range of artists and contributed to countless successful albums across various genres.
Marc Matthews:
And today we'll be discussing the intersection I've got written here in this elaborate introduction I've got of mastering and artificial intelligence.
Marc Matthews:
Now this is sort of like part of a mini series I've got going on, so at the point of this episode going live, a previous episode would have been with Bobby Osinski about his book AI in music production as well. So it's a nice little mini series. So really excited for this one and I was saying off air as well that your Are you Listening series is probably my most signposted suite of content that I send the listeners to when they ask me questions where I'm kind of like, actually you know what I could give you the answer, but Jonathan probably puts in a much more palatable way than I do. So, uh, yeah, very much so, and they probably heard me mention it a few times on the podcast, so I thought it'd be quite good if we can kick off with so you mentioned about the development of audio technology and whatnot If you could talk about how you see artificial intelligence influencing the mastering process and, in particular, what are some common misconceptions about AI and mastering that you often encounter.
Jonathan Wyner:
Well, I've never actually heard anybody ask about how AI might influence the mastering process. Anybody ask about how AI might influence the mastering process. You know, I think a lot about the way technologies as they come across our desks actually change not only our workflows and the way we do things, but also the aesthetics of what we do, and there's some famous examples of that, going back through the ages, whether you know, especially in the introduction of digital signal processing, around the introduction of limiters and even being able to use a buffer, like once buffers became affordable in computers so that we could hold on to a signal for a moment, analyze it, figure out what the pitch was, figure out something about the signal compute, sort of the low frequency period of a signal.
Jonathan Wyner:
You can't do that in analog, you can only do it in digital, and that resulted in a complete sea change in terms of the aesthetics of sound. So, anyway, you know, I'm not sure I have a single answer to how AI will affect the aesthetics, but I can guarantee you that it will. One of the things that first comes to mind is the ability to engage in source separation, which is, at this point, I think, probably everybody is familiar with this idea of demixing. You can take a full mix and separate it into four stems or maybe more. Audioshake and some other platforms are extending the vocabulary. And then what we do with that information is fascinating and varied, and more than simply doing karaoke or remixing. But we can take the signals that are extracted and use them as side chains to feed different signal processors in our mastering chains or in our recording or mixing production.
Jonathan Wyner:
I think that there's a lot of sort of interesting innovation that falls out of simply having access to components in a mixed signal. In that way, you could tune your vocals as you're mastering. I mean, that's pretty mundane. Now here's sort of another take on this, and I'm gonna say this this may sound a little bit I'm going to say a little bit harsh, but I think it's something that we all need to really acknowledge and embrace, and that is so we can talk a little bit about what AI and mastering actually means, which was the second part of your question.
Jonathan Wyner:
But I think we all have to allow for the fact that on some very sort of superficial level, ai-driven tools in mastering may do a reasonably good job Now, maybe not full of creativity and interesting results as a human, but let's just say sort of baseline. It's a competent kind of processing, depending on how models are trained et cetera. So now let's take a look at. You know, the mastering marketplace has been exploding over the last bunch of years with the advent of ozone and other approachable tools.
Jonathan Wyner:
We have more and more people coming into the market who have relatively little experience, and it takes a while to get good at something. And so you may see where I'm going with this. But if the entry level can't measure up or doesn't measure up to what the AI-driven tools can do, that may exert some pressure on the market in general. It may sort of further dilute the market. It may mean it's more difficult for people to enter the market. So just in terms of the activity, I think there's potential for AI to have an impact and maybe to encourage people to look at what it's doing and make sure you can do at least as well as what the best of AI-driven tools are.
Jonathan Wyner:
Now I assume we'll get to the question of what the AI sort of in mastering or any other AI tools do and don't do well, at least currently. But let's just acknowledge that there's certain things that may be where the tools may be competent Certain kinds of ways of adjusting signals, understanding signals and I'm by no means an AI maximalist, right, I'm not saying you know, the robots are coming to take our jobs and they're going to take over and all of our pets.
Jonathan Wyner:
And you know, social life is going to be AI in five years. So let's take a look at what AI and mastering actually is. And I'll start by saying that probably the celebrity of the AI and mastering world is Lander. So Lander is a company that was started probably 10 years ago, maybe a little bit more, and the original registered trademark was MixWizard, and so many people are surprised to find out that what the intention of the platform originally was was to develop an auto-mixing environment.
Jonathan Wyner:
It became very evident very quickly that mixing is hard, and creating a mixing environment driven by machine learning and we should differentiate between machine learning and AI that produced decent results probably wasn't going to happen very quickly. So they pivoted to mastering, because in some ways, mastering on the surface of it is a much simpler thing to understand. You know there are a few things. Whenever you ask anybody what happens in mastering, the thing that they will probably say is it's where our projects go, to get loud to be made loud, which is a proxy for setting level and then probably to be made brighter right, even though that's not necessarily the thing you want to have happen.
Jonathan Wyner:
That's what people think about mastering. So if you take those two very high level concepts, you know, setting the level and getting the tone, which is kind of a two-dimensional measurement across an entire program, then you could say, well, sure, you could measure level. That's pretty easy. You can measure tone. You can take kind of an FFT average across a certain amount of time in a program and then you can compare it against an average that's created via machine learning yeah, all right, sort of data mining and say, okay, so this is how that varies. From that. We'll make an adjustment, you know, we'll set the level differently, probably make it hotter. We'll do some kind of EQ, maybe some kind of dynamics processing, in order to change the dynamism, either broadband or within parts of the spectrum, and that's going to be mastering.
Jonathan Wyner:
And then, beyond that, some of the tools have now started to try to either give the users options driven by semantic sort of attributes you know, a soft versus an aggressive version, you can check a box or, in the case of the work that we did at iZotope, we tried to use genre tags as a way of designating certain kinds of tonal curves and treatments, which is interesting.
Jonathan Wyner:
It just creates a little more nuance in the result. But at the end of it all, it really is what I just said it's level and tone, and it could be more or less automated. There are certain platforms that fully automate it, like put in your track, hit, go, you get something back, like it or not. Here you go, and then there are other tools. I'll sort of take it all the way back to iZotope and Ozone, where there's an assistant that produces a treatment that you can simply accept, but it also lets you unpack it with as much detail as you'd like. So you know, to the point where you can go in and change the peak detector to an average detector and the compressor. You can moderate and modify any of the parameters to your heart's content. So there's the automated version of this kind of tool and then there's the assistive or, you know, your assistant.
Jonathan Wyner:
I think, is the term that we used to use and still is used by many tools and certainly iZotope, so hopefully that's a pretty good sort of overview of what AI and mastering means. You know what it doesn't mean? We can talk about that too.
Marc Matthews:
Yeah, yeah, fantastic, yeah, just to recap some of the bits you went through there in particular. So you mentioned about the source separation, which I think is really interesting, because it's the same conversation I had with bobby azinski in a previous episode and we mentioned the beatles, or rather he mentioned the beatles film, whereby they separated the mix there and they were actually able to separate the drum stems that weren't originally recorded separately, as it were, so they were able to separate the kick snare. I might be doing a crude description of it, but I think that's incredible being able to do that, because I know I've had instances where I've been sent tracks whereby there needs to be something changed level-wise before it hits the master and the client has said I no longer have that project available, I haven't got access to it anymore, which comes down to project management, but that happens a lot.
Marc Matthews:
So that I think is incredibly useful. And it's interesting what you mentioned there about sort of like the barrier to entry with mastering as well, with these products being available, and do you think then Could it potentially, if you've got the facilities there to have, like you described there, with a mastering assistant, would that then mean there could be more? Is the barrier to entry lower then for mastering engineers to enter the market because they've got this assistant and then they can learn on the job? Would that be a fair description?
Jonathan Wyner:
Yeah, there's probably three answers to that. I want to go back to the source separation again for one second Go ahead.
Marc Matthews:
We're going to have parallel conversations or one second.
Jonathan Wyner:
Yeah, go ahead, we're going to have parallel conversations or interleaved conversations, I guess. So the sort of isolating drums and low frequency instruments from other tonal instruments. At this point that's become kind of a relatively simple task.
Jonathan Wyner:
The thing that was fascinating about the Beatles example, and the place where the vocabulary of these tools is getting extended, is being able to separate voices. So being able to separate John's voice from Paul's voice, now that. Or taking a four-part harmony and being able to deconstruct it, so you've got the soprano, the tenor, the, you know whatever, and I think that's the direction we're moving into. So it's no longer anyway, it's becoming more capable and more subtle and more nuanced, and so that's that. I just wanted to sort of feed into that.
Marc Matthews:
Yeah, of course.
Jonathan Wyner:
So in terms of access I mean. So I'm going to take your last point about learning. I think one of the greatest benefits about this technology is that, with an open mind and with a spirit of inquisitiveness, you can sort of look at what these tools are doing and, assuming that they are informed by a good data set and that's an assumption, yeah, we can dive into that too, but, assuming it's informed by a good data set observe the outcome and then say, oh, I see, so here's what I've been doing, or here are my mixes and this is what these systems are proposing all the time. So let me sort of see what I can make of that information. You know, my mixes are always a little dull, or my levels, you know, in a good place, not in a good place. Or you know, it seems like the low end of my kick drum is always interacting in a negative way with these tools. Maybe I should go back and rethink my mixing so they can provide some insight into the user's work and in that sense it really is kind of a neat assistive technology.
Jonathan Wyner:
In terms of accessibility. I guess it's a double-edged sword because on one hand, yeah, you know in the same way that, like I don't know. If you remember, there was something made by TC Electronics called the Finalizer, which is like a mastering engineer in a box. It was one of the earliest hardware sort of mastering wizard things no AI, but it had a multiband compressor, an EQ, a reverb and a widener and you know, it was instant access to mastering tools For mastering process. You just push a button and suddenly for at that time it was probably $1,100, you had access to this. Now I've actually got an extra one. If you'd like I'd send it to you.
Marc Matthews:
Oh, yes, please, I'd love to try it out. That would be amazing.
Jonathan Wyner:
They were pretty funny devices. So anyway, the sort of access to the tools for mastering has sort of accelerated. You know, through Ozone through there was something called T-Rex that was made by IK Multimedia. That was, I think, probably earlier than Ozone. We're right around the same time that it came on the market. So that's provided greater access. And now AI sort of does two things at once it speeds up the workflow, it does increase access, but it can also be more opaque. So the learning that you take away from using something like Lander is a little harder to come by. You have to make your own observations and make your own deductions With something that's assistive, that unpacks the processing in front of you. Then you can say oh, now I hear what I hear and I see why I hear it, and then I can sort of get a little bit of that insight more directly from the feedback.
Marc Matthews:
Yeah, it's kind of like reverse engineering, isn't it? I think I've said that before on the podcast is where you've got these tools and access to them. You say, okay, well, it's made that decision, how has it got to that decision? And then I can reverse engineer it from there and understand and unpack what's happened, whereas I guess, like you say, with a platform like Lander or possibly CloudBounce as well, you kind of like it just spits out the end product and you don't necessarily know how it's got there. That's right.
Jonathan Wyner:
There are 22 online mastering services at the moment.
Marc Matthews:
Wow, are there really?
Jonathan Wyner:
I did not know that 22 online mastering services.
Marc Matthews:
That's right, separate and distinct kinds of processing engines. I'm just going to make a note of that 22. Separate and distinct kinds of processing engines I'm just going to make a note of that 22.
Jonathan Wyner:
I'm going to go and do a bit of research into it, because I did not know that and that was as of yesterday. There might be more today.
Marc Matthews:
Do you think, then, this is going off on a tangent? Then you mentioned about Landa starting out as an auto-mixing service. Do you think that that will eventually be something where we upload stems for want of a better way of putting it, Stems would be the right way and then it mixes it for us? Do you think that's something that's on the horizon? Oh, it's already happening. Is it really?
Jonathan Wyner:
Yeah, there's a platform called Roex, started by a fellow named David Ronin, another one called Osmix OSmix in the market and actually at iZotope. We tried to sort of put something together that was a mixing assistant within the context of the Neutron plugin.
Marc Matthews:
Yes, another one.
Jonathan Wyner:
And so absolutely, and you know, I think, for the purposes of this discussion, I just want to and you know, I think, for the purposes of this discussion, I just want to sort of state a sort of a focus for us, and that is that we are talking about all of this technology in the context of bespoke music production to the sort of writing for commercials or advertising, where kind of good enough means something very different than it does if you're trying to make music that makes people happy and inspires their imaginations, as opposed to selling products. Because you know, I just want to sort of say that at this point so we can not go into the yeah, auto mixing is good enough for the people who just want a 30 second spot that starts slow, ends up fast and sounds like reggae or something like that, because those engines already exist. Back to the sort of auto mixing idea. I think that we that the learning of the systems is improving, it's getting faster and there's sort of improvements that are iterative, over time. You know, if you train a system long enough, it gets better. Yeah, yeah, you know the difference between training like a system, a machine learning system, for one hour versus one day, versus three days versus a month is profound.
Jonathan Wyner:
So, having said that, one of the big problems with auto mixing systems is the user experience, the design of the system, and I'll just illustrate a couple of problems that you get. First of all, you have to tell the system what the focus of a mix is. And if there's drums, bass and vocals, sure it could assume those things, but what if it's an instrumental track? Or what if in a section there is no vocal any longer? Or what if you have some other idea about what should be the priority of a mix? So initially you have to give the system some guidance, and that requires user input. So that already creates a layer of interaction that is complicated.
Jonathan Wyner:
And then, if you think about, you know, if you've got your multi-track environment, where you've got 60 or 70 or 80 tracks, you have to wait for the system to scan everything, ingest everything, identify everything. Hopefully it's correct, hopefully it's grouped them in the way that you want to group them. So there's a lot of like pre-work for the system to do to get to the point where you can even make use of it. And then, how does that integrate into your particular DAW? Most DAWs are not yet willing to bring this into their product environments, probably to protect the IP, probably to protect their market and probably because it's a lot of work, the ip probably to protect their market and probably because it's a lot of work, um. So we're a ways away, I think, from it being uh sort of commonly used um and in use, but but inevitably I think it will be yeah, it's interesting what you said there about how you.
Marc Matthews:
Obviously there is that layer of interaction whereby, essentially, we are having to prompt it to do what we want it to do, and then it comes down to whether or not we get the prompt right. And uh, I've noticed this with generative ai, because I use generative ai and I like to experiment with these different bits and pieces, and if you don't prompt it correctly, then you're not going to get. You'll get it, you'll get it, you'll get an output, but it won't logically. I'm going down the computer science route now, but it won't logically. I'm going down the computer science route now, but it won't logically be correct, it won't be quite what you're after. So we've almost got to learn another skill set now, which is how good we are at prompting computers to do what we want them to do. Is that a fair assumption?
Jonathan Wyner:
Absolutely, and the engineering of the systems. There has to be some agreement about language and mapping the language to the sound examples. You've heard this term multimodal systems, which is environments that describe the ability to work not just with semantic prompts, but also having either video or images or audio examples. A lot of the LLMs that are in use right now have never listened to anything. They've never heard a sound, and so mapping the language to the sound that you're after is not a simple task. It's hard to get it right.
Marc Matthews:
Yeah, very interesting. I cannot wait to see what it looks like in five years' time, bearing in mind how far we've come in the previous five years in terms of what every platform now has this ai component, because I think there was a clamor for it. No, they're not just in audio, but in in video as well, in imageries and and every. All these platforms and I have this platform using right now to to on this podcast riverside they uh, when I started using there was a really basic element.
Marc Matthews:
If not, there might not have been any AI integration. There probably was, but now it's just a hockey stick curve in terms of what they're doing, which is amazing. Jonathan, in the interest of time I'm well aware we're already 25 minutes in I think it'd be quite interesting to now jump on what you mentioned earlier about what AI mastering can and cannot do for us. I think it'd be quite interesting to now jump on what you mentioned earlier about what what ai mastering can and cannot do for us. I think it'd be quite cool if you could talk about that and how well, basically, what it can and cannot do for us and how it can assist us as creatives.
Jonathan Wyner:
Sure well, I mean, I think, both for mixing engineers and for um sort of those who are learning or coming into the marketing I'm sorry, the mastering market, the activity. As I said, it can give you some guidance and that's a great use of it. It can also, you know, for somebody who's creating an album of demos and you just want to get everything into a place where you could send it out for somebody, a producer, to listen to or something at a label to listen to. You know, it's kind of an easy win. You know the problems that have not yet been addressed are how do you indicate intent, how do you understand musical context and how do you facilitate the sort of interesting and creative things that one does in mastering?
Jonathan Wyner:
When you're interpreting a mix and you get a sense of what you think the artist, what the vision might be, and you take it in a direction, often that decision is informed by lots and lots of information. It's not just about level and tone, and sometimes you come up with an idea to do something that's slightly unconventional. And sometimes records that don't sound perfect or don't conform to the model are the most interesting records you know. Probably the best most recent example is the Billie Eilish's record two records ago, which was very different sounding from pretty much anything else on the market. I don't think a mix, an AI mix engine, would have mixed it the way they mixed it, and I don't think an AI mastering system would have mastered it the way they mastered it and, frankly, it's actually got a little too much base in it. You know, from a technical standpoint it ain't correct, but it's really great and it's really cool and it's you know. One can't argue with the commercial success. No, not at all.
Jonathan Wyner:
No. So Drilling down a level no pun intended. You know the nuances, such as the difference in the level between an introduction and a first verse, or a verse and a chorus, being able to sort of program, a system to assess that difference and then make a change that would actually be consonant with what was desired, which is one of the things that sometimes we do in mastering. You want to maintain the impact from the intro to the first verse. There's an example of this that the first experience that I had with this is when I was mastering a record. This is probably seven or eight years ago for my daughter. It was in a punk rock band and it started with a really janky guitar intro and then the drums explode after this four bar intro and I had a few years later I decided to use it as an example and sample and sent it to a couple of engines, ai driven mastering systems, and all of them completely obliterated the contrast, destroyed it. You know, suddenly I mean they did a great job of matching the level by compressing the heck out of it, because probably they measured too much dynamic change across either some part of the mix or the whole mix. You know that lacked all the context, for you know what was built into the mix, so that that's a problem right. And how do I mean, you know, how do we make these systems in such a way that they actually can can sort of take that kind of consideration into account?
Jonathan Wyner:
Well, there's another sort of whole arena that I think requires greater exploration and that is around genre, and I know that, as I said earlier, at iZotope we used genre tags to try to give people a way to give input and curate the results a little bit differently. But frankly, I think genre as a word is very hard for AI to actually wrap its artificial brain around. I think style transfer and style is something that's easier to understand. You know, if you were to describe what makes disco disco, you'd probably talk about the level of the hi-hat and the, the snappiness of the drums and this. You know, the tone of the bass, and there are very specific attributes that you could define, um, but what makes something kind of a, a disco dance, hit from the standpoint of a genre, is sort of a very different construct. And then other genres, like, involve culture and sort of much deeper concepts that I think it's very hard for us to reduce them to the kinds of features that are easy to measure and quantify and build into a database.
Marc Matthews:
So those are all some areas where I think the AI and mastering could improve what you've mentioned there right at the beginning about how you could use it for guidance and demos sort of resonates a lot with what the conversation I've had on this podcast over the last hundred plus episodes in which I've spoken to producers, artists, and they say, yeah, for example, logic just at the end of last year introduced the mastering assistant into logic and it's a way of just okay, well, what could it sound like? I'm mixing at the moment, what could it sound like, inevitably, and it just gives you those guidelines. But I think I totally agree with what you say about with the mastering and the engineer element of it. And going back to that billy eilish record, and in a way, sometimes you get those happy accidents that you do. You get out of mixing as well. You do something. You're thinking actually I didn't mean to do that, but it sounds really good and you're just not going to get that from artificial intent at the moment. You're not going to get it from our.
Marc Matthews:
With the growth mindset there, I'm saying you're not going to get it yet. Let's say um, but I suppose that's what it comes down to genre, because I was speaking to someone earlier today and they were saying, um, can you help me pinpoint what genre I am, because they didn didn't know they were. Like I've had someone say it's this, someone says it's this, someone says it's that and I don't really know what genre of music this is. So I guess once again, it comes down to being able to prompt correctly and that sort of feeds into what you said about the genre discussion around mastering and how it's not quite there yet. I suppose that'd be fair to say.
Jonathan Wyner:
Yeah, I suppose that'd be fair to say. Yeah, that's right. I mean, I think, defining genre, defining culture using a, I mean, I really think that there's a cultural component in all of this and I you know it's especially true of genres like jazz or certain sort of what we would call world musics, where there's either harmonic vocabulary or rhythmic vocabulary or even the role of individual elements. That's very different from probably what's represented by most of the data sets, you know, which actually parenthetically brings up the whole question about bias and data. You know, if all of the records that you feed into a system have some similarity to them, chances are that that can be both a strengthness but also a blind spot or a weakness in a machine learning system.
Marc Matthews:
Yeah, very interesting it really is. In a previous life I was a teacher of computer science, so this is why it's all very interesting to me. When you mentioned there about bias and the whole idea about randomization in computing as well, where it's pseudo-randomization and things like that, and well, you can go down a total rabbit hole in that instance, you know.
Jonathan Wyner:
But I'll give you a very specific example of where this showed up, which was when we were training the vocal assistant for Nectar, which is another isotope product, and after some I think it was a couple of days of learning we started to recognize that the system observed accurately that every vocal that was fed into the system was in tune, so it assumed everything needed to be tuned and that was a bias that was built into the data which was not intended. So we had to start again and kind of make sure that we removed that as a feature if you will, interesting.
Marc Matthews:
It's amazing that when you hear the stories behind the scenes, under the hood of how it was all put together, because what we see as consumers is this great piece of kit, but you don't realize all the work and the dev work that's gone into it and all those bits and pieces, which I can imagine is quite a feat to do. Hey, listen, while you said that.
Jonathan Wyner:
if I may, a 10-second plug, oh please. In June of this year we're hosting an AI and the Musician Symposium in Boston, massachusetts, at the Berklee College of Music, partly just to give musicians access to the kinds of thinking that's going into the design of tools that you're describing, so I just wanted to mention that Not everybody's going to travel to Boston in June, but if you happen to be in the neighborhood, please attend I.
Marc Matthews:
I had this conversation with uh, with matt um off air who gives the warm introduction, and he mentioned it to me and I was like june and at the point it was february. I was like that's four months I might be able to put something together and get over to boston. That would be amazing. I won't lie, that'd be a nice little trip for me. The uh, I just I won't tell the girlfriend. So we're going to go to boston look, it's a.
Jonathan Wyner:
It's a great place to visit, in june also. I'll tell you imagine um so yeah, fantastic.
Marc Matthews:
Um, and audience listening. I'll put a link to that and a bit more information in the episode description as well. Um, so you can go and check that out, if you are. I know we've got. I want to say a sweeping statement here, but I think a lot of other listeners are in the united states. So, um, yeah, yeah, yeah, which is, maybe they like the english tone or not quite quite sure it could be something like that.
Marc Matthews:
Um, yeah, I'll have to try that sometime uh, jonathan, we're coming towards the end now, so I think it'd be quite nice just to maybe wrap things up with. If we've, maybe you could talk a bit about, if you've got an artist who's navigating this landscape of sort of AI mastering versus professional mastering services, maybe what sort of considerations they should take into account if they're thinking, if they're in, if they're on the fence, do I go with AI mastering or do I go with what's your single? Maybe it'd be what's your biggest piece of advice there.
Jonathan Wyner:
Well, what I'm about to say absolutely reflects my own bias and my own values, and it's not just driven by the sort of creative economy either. But you know, when you make a record a year later, what are you going to remember about that record? Are you going to remember, you know, that you spent $500 doing this or $1,000 doing that, or are you going to remember the ways in which the record succeeded, whether it's commercially or in terms of the artistic vision? If you look at it through that lens, you can tell that I'm advocating for the bespoke approach. And in the right interaction, you also stand to learn more there still, because if you're collaborating with someone, you can get feedback, there's an iterative process, and so for all of those reasons, I would absolutely advocate for the sort of human collaboration.
Jonathan Wyner:
Yeah, if you're somewhere and I want to sort of say this without it sounding too judgmental, but over here is kind of the music you're making for today and you just want to get something out the door and test it in the market, and then there's the thing over here, which is the thing that potentially has some legacy for you, if you're more on this side of the spectrum, then sometimes it may make sense to just throw a track up and make sure it comes back.
Jonathan Wyner:
It doesn't sound too bright, it's not been squashed too hard and you can put something out and it's less expensive, it takes less time to do that.
Jonathan Wyner:
I mean you can get it back in 10 minutes instead of waiting for 10 days to book somebody, and that may be exactly the right thing to do. So there's some of you know there's some gray areas in between. It's not a fully binary scenario, but you know again, as much as I love doing this work and as much affection as I hold for all of my peers who are amazing mix engineers and mastering engineers, I also recognize that there's a real pressure on the creative economy for artists, and you can make an argument that if you make something that sounds great, it's more likely to succeed commercially. But you can also make an argument that it's hard for artists to make money and so you have to be careful and smart about where you spend it. So you can tell I'm not um, I'm not recommending the ai version because it's going to be better in any instance, but I understand when sometimes it might be good enough I suppose it comes down to intent and situation, I guess, in a way like what, what is it?
Marc Matthews:
what is it? Because I think it goes back to the clarification that you mentioned earlier in this episode, whereby it was are you creating music that you want to get out there quickly for me, for example, um tv and film or something along those, some sort of sync opportunity in that respect, that's right or are you or are you trying to create something that's, as you mentioned, legacy? So I guess it really does depend on what the and also budget, like you say, artists and budget, and it comes down to that as well. So I suppose it's quite a tricky question, isn't it? I mean, there are many factors involved with regards to what it is you actually want to do with the music that might well will influence your decision.
Jonathan Wyner:
Yeah, I mean I could say snarky things like my clients are people who actually care about their music, but I'm not going to say that, even though I just did. You know, I mean it's kind of true. But, that's not the only way.
Marc Matthews:
Jonathan, before we wrap things up, I just want to. We mentioned this off-air about an Easter egg in this episode, and that was with regards to your T-shirt. So, for those of you watching this on YouTube, because it's a classic album cover, isn't it? Obviously, there are four cats on there at the moment as well If you can identify that album cover, please do write it in the comments and if you listen to this on your podcast player of choice, head over to YouTube and check it out and see if you can figure out what it is. Uh, jonathan, before you go, you've already mentioned that about what's happening in june, but if our audience, I want to find about a bit more about you and your past, what you're doing at the moment. Uh, where should they go online?
Jonathan Wyner:
well, my, my mastering studio and, by extension, um other things audio is called M-Works Mastering. We're M-Works Studios here in the Boston area, specifically in Somerville, massachusetts. You can find me both in person and online at Berkeley College. I've written a couple of mastering courses for the online school and also teach at the Brick and Border School in Boston and also teach at the Brick and Border School in Boston and I usually find my way to AES and NAMM, and you know this. Last year I was speaking in Norway, in Japan, and I think those were my two big trips recently. But you know I travel around sometimes and show up at schools and give talks, which is something I enjoy doing very much. Have you got any plan for the UK? You know, maybe I was just talking with some folks about doing some work in London in the next couple of months.
Marc Matthews:
And if I?
Jonathan Wyner:
do? I might end up at BIM, or we'll see.
Marc Matthews:
Yeah, that'd be amazing. I'll keep an eye out. I'm due for a London trip. I think I'm going there this is slightly off topic, but for a podcast what do they call it? Fair I? Don't think it's fair. Convention. That was it.
Marc Matthews:
That's what I was looking for Convention Confair yes, yeah, yeah, something like that. Jonathan, it's great to meet you as well. As I mentioned, you have been referenced on this podcast many a time, so it's great having you here today and I will catch up with you soon. Thanks very much, mark. I appreciate it.