SteerMusic: Enhanced Musical Consistency for Zero-shot Text-Guided and Personalized Music Editing

Abstract

Music editing is an important step in music production, with broad applications including game development and film production. Most existing zero-shot text-guided editing methods rely on pretrained diffusion models involving forward-backward diffusion processes. However, these methods often struggle to preserve the musical content, while text instructions alone usually fail to accurately describe the desired music. We propose two music editing methods that improve consistency between the original and edited music by leveraging score distillation. SteerMusic is a coarse-grained zero-shot editing approach using delta denoising score. SteerMusic+ enables fine-grained personalized music editing by manipulating a concept token representing a user-defined musical style. Experimental results and user studies show that the methods improve music content consistency, editing fidelity, and overall editing quality.

Michele Mancusi

Senior Applied Scientist, PhD

Senior Applied Scientist at Music.AI (Moises), developing and optimizing autoencoders and generative models for speech, music, and general audio enhancement