AES Show 2024 NY

Students! Joins us so we can find out where you are from, and tell you about all the exciting things happening at the convention.

The upcoming 2025 AES International Conference on Artificial Intelligence and Machine Learning for Audio (AIMLA) aims to foster a collaborative environment where researchers and practitioners from academia and industry can converge to share their latest work in Artificial Intelligence (AI) and Machine Learning (ML) for Audio.

We want to advertise the upcoming AIMLA at the AES Show, to encourage early involvement and awareness from the AES community. To better accommodate the central themes of the conference, we propose new additions to the typical AES proceedings, such as challenges and long workshops, that can more appropriately showcase the rapidly growing state of the art. In this presentation, we plan to give an overview and a discussion space about the upcoming conference, and the changes we want to bring into play, tailored for AI/ML research communities, with references to successfully organized cases outside of AES. Finally, we propose a standardized template with guidelines for hosting crowdsourced challenges and presenting long workshops.

Challenges are a staple in the ML/AI community, providing a platform where specific problems are tackled by multiple teams who develop and submit models to address the given issue. These events not only spur competition but also encourage collaboration and knowledge sharing, ultimately driving forward the collective understanding and capabilities of the community.

Complementing the challenges, we introduce long-format workshops to exchange knowledge about emerging AI approaches in audio. These workshops can help develop novel approaches from the ground up and produce high-quality material for diffusion among participants. Both additions could help the conference become an exciting and beneficial event at the forefront of AI/ML for audio, as they intend to cultivate a setting where ideas can be exchanged effectively, drawing inspiration from established conferences such as ISMIR, DCASE, and ICASSP, which have successfully fostered AI/ML communities.

As evidenced by the recent AES International Symposium on AI and the Musician, we believe AI and ML will play an increasingly important role in audio and music engineering. To facilitate and standardize the procedures for featuring and conducting challenges and long-form workshops, we will present a complete guideline for hosting long-form workshops and challenges at AES conferences.

Our final goal is to promote the upcoming 2025 International Conference on AI and Machine Learning for Audio, generate a space to discuss the new additions and ideas, connect with interested parties, advertise and provide guidelines regarding the calls for crowd-sourced challenges and workshops, and ultimately get feedback from the AES as a whole to tailor the new conference to the requirements of both our AES and the AI/ML communities.

The presence and hype associated with generative AI across most forms of recorded media have become undeniable realities. Generative AI tools are becoming increasingly more prevalent, with applications ranging from conversational chatbots to text-to-image generation. More recently, we have witnessed an influx of generative audio models which have the potential of disrupting how music may be created in the very near future. In this talk, we will highlight some of the core technologies that enable novel audio content creation for music production, reviewing some seminal text-to-music works from the past year. We will then delve deeper into common research themes and subsequent works which intend to map these technologies closer to musicians’ needs.


We will begin the talk by outlining a common framework underlying the generative audio models that we will touch on, consisting of an audio synthesizer “back-end” paired with a latent representation modeling “front-end.” Accordingly, we will overview two primary forms of back-ends in the forms of neural audio codecs and variational auto-encoders (with examples), and illustrate how they pair naturally with transformer language model (LM) and latent diffusion model (LDM) front-ends, respectively. Furthermore, we will briefly touch on CLAP and T5 embeddings as conditioning signals that enable text as an input interface, and explain the means by which they are integrated into modern text-to-audio systems.

Next, we will review some seminal works that have been released within the past year(s) (primarily in the field of text-to-music generation), and roughly categorize them according to the common framework that we have built up thus far. At the time of writing this proposal, we would naturally consider MusicLM/FX (LM), MusicGen (LM), Stable Audio (LDM), etc. as exemplary candidates for review. We will contextualize these new capabilities in terms of what they can enable for music production and opportunities for future improvements. Accordingly, we will draw on some subsequent works that intend on meeting musicians a bit closer to the creative process. At the time of writing this proposal, this may include but is not limited to ControlNet (LDM), SingSong (LM), StemGen (LM), VampNet (LM), as well as our own previous work, as approved time permits. We will cap off our talk by providing some perspectives on what AI researchers could stand to understand about music creators, and what musicians could stand to understand about scientific research. Time permitting, we may allow ourselves to conduct a live coding demonstration whereby we exemplify constructing, training, and inferring audio examples from a generative audio model on a toy data example leveraging several prevalent open source libraries.


We hope that such a talk would be both accessible and fruitful for technologists and musicians alike. It would assume no background knowledge in generative modeling, and may perhaps assume only the most notional conception as to how machine learning works. The goal of this talk would be for the audience at large to walk out with a rough understanding of the underlying technologies and challenges associated with novel audio content creation using generative AI.

Every year, the availability and capabilites of processors and sensors expand greatly while their cost decreases correspondingly. As an instrument designer, the temptation to include every possible advance is an obvious one, but one that comes at the cost of additional complexity and probably more concerningly, reduced *character*.

Looking back on electronic instruments from the past, note that what we love about the classics are the quirks and idiosyncracies that come from the technical limitations of the time and how the designers ended up leveraging those limitations creatively. This panel will bring together designers to discuss how we are self-imposing constraints on our designs in the face of that temptation to include everything that's possible.

As Brian Eno said: "Whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8-bit - all of these will be cherished and emulated as soon as they can be avoided."

Panel Description:
Join us for an in-depth, candid discussion with prominent NYC studio owners on the state recording industry of today and where it might go tomorrow. This panel will explore trends in studio bookings, predictions for the future, and the evolving landscape of recording studios. Our panelists will share their experiences of owning and managing studios, offering valuable insights for engineers on essential skills and best practices for success in today’s recording environment. We'll also discuss the importance of high-quality recording, and how these practices are crucial in maintaining the soul of music against the backdrop of AI and digital homogenization.

Topics also include:
What are studios looking for from aspiring engineers, and what does it take to work in a major market studio?
Studio best practices from deliverables to client relations
To Atmos or not to Atmos?
Proper crediting for engineers and staff
Working with record labels and film studios
Can recording and mix engineers find the same labor protections as the film industry?

Panelists (in alphabetical order)

Amon Drum, Owner / Engineer, The Bridge Studio (https://www.bridgerecordingstudio.com/)
Amon Drum is a recordist, producer, acoustic designer, owner, Chief Engineer, and designer of The Bridge Studio in Williamsburg, Brooklyn. He has an expertise in both analog and digital recording techniques. Amon specializes in the recording of non-western acoustic instruments, large ensembles, and live music for video. Amon is also a percussionist, having trained with master musicians including M’Bemba Bangora and Mamady Keita in Guinea West Africa, and brings this folkloric training to his recording & productions. Working with a wide variety of artists and genres from Jason Moran to Adi Oaisis, Run The Jewels, to Ill Divo as well as many Afro-Cuban ensembles from the diaspora.

Ben Kane, Owner, Electric Garden (https://electricgarden.com/)
Ben Kane is a Grammy Award winning recording and mix engineer, producer, and owner of Electric Garden, a prominent recording studio known for its focus on excellence in technical and engineering standards, and a unique handcrafted ambiance. Kane is known for his work with D'Angelo, Emily King, Chris Dave, and PJ Morton.

Shahzad Ismaily, Owner / Engineer, Figure 8 Recording (https://www.figure8recording.com/)
Shahzad Ismaily is a Grammy-nominated multi-instrumentalist, composer, owner and engineer at Figure 8 Recording in Brooklyn. Renowned for his collaborative spirit and eclectic musical range, Shahzad has contributed his unique sonic vision to projects spanning various genres and artists worldwide.

Zukye Ardella, Partner / Engineer, s5studio (https://www.s5studiony.com/)
Zukye is a New York City based certified gold audio engineer, music producer and commercial studio owner. In 2015, she began her professional career at the original s5studio located in Brooklyn, New York. Zukye teamed up with s5studio’s founder (Sonny Carson) to move s5studio to its current location in Chelsea, Manhattan. Over the years, Ardella has been an avid spokesperson for female empowerment organizations such as Women’s Audio Mission and She is the Music. Talents she’s worked with include NeYo, WizKid, Wale, Conway The Machine, A$AP Ferg, Lil Tecca, AZ, Dave East, Phillip Lawrence, Tay Keith, Lola Brooke, Princess Nokia, Vory, RMR, Yeat, DJ KaySlay, Kyrie Irving, Maliibu Mitch, Flipp Dinero, Fred The Godson, Jerry Wonda, ASAP 12vy and more.


--

Moderator:

Mona Kayhan, Owner, The Bridge Studio (https://www.bridgerecordingstudio.com/)
Mona is a talent manager for Grammy award winning artists, and consults for music and media companies in marketing and operations. She has experience as a tour manager, an international festival producer, and got her start in NYC working at Putumayo World Music. As an owner of The Bridge Studio, Mona focuses on client relationships, studio operations, and strategic partnerships. She also has an insatiable drive to support and advocate for the recording arts industry.

In this timely panel let’s discuss some of the pressing matters AI confronts us with in audio, and how we can turn a perceived foe into an ally.
We’ll discuss issues including:
During production, AI cannot deal with any artistic issues related to production and engineering, most of which depend on personal interaction - as well as perception.

In post-production, AI could be of use in repetitive tasks: making a track conform to a click-track and maintain proper pitch, perform pitch correction on a track, deal with extraneous clicks (without removing important vocal consonants), perform ambience-matching (particularly on live recordings), to name a few. Can AI running in the background on our DAW build macros for us?

The more we can use it as a tool for creativity and enhance our revenue streams the more it becomes a practical, positive approach. Many composers are using it to create or enhance their musical ideals almost instantaneously. The key here is that it is they are our ideas that AI adds to.

How do we embrace AI, adapt to it, and help it to adapt to us? Can we get to the point where we incorporate it as we have past innovations rather than fear it? How do we take control of AI instead of AI taking control of us?

What should we, the audio community, be asking AI to do?