AI写歌终于能改谱子了:先写乐谱再录音
以前AI做音乐是两条路:要么只写谱子,停在干巴巴的MIDI;要么直接出成品歌,但你想改个和弦、换个旋律,它听不懂。YuE2把这两件事打通了:同一个模型先写出一份人能读懂的乐谱,再把它变成完整歌曲。好处是,你终于能对AI说「第二段副歌升个调」,它改的是谱子,而不是整首歌重来。专家盲听里,49.3%的人更喜欢这种先写谱再生成的方式,胜过直接生成。它还能做零样本翻唱——没专门训练过,给首歌就能换个风格唱。这不是你明天就能用的工具,但它指向一个方向:AI音乐从「碰运气生成」走向「可修改的创作」。
📄 原文摘要(英文)
Symbolic models make melody, harmony, rhythm, and form explicit but typically stop before a finished recording; audio models produce complete songs while leaving composition implicit. We introduce YuE2, which unifies symbolic and audio music generation at frontier quality through symbolic planning. A single AR-NAR Mixture-of-Transformers (MoT) first writes a readable score specifying melody and harmony, expands it into semantic music tokens, and realizes it as full-song audio. In comparisons using the same checkpoint, experts prefer symbolic planning for overall quality and musicality, with 49.3% of overall preferences versus 34.6% without planning. Experts also favor the unified model over a separate language model and diffusion Transformer. On WildSongBench, YuE2 scores 6.73 on SongBench Global Avg, exceeding all evaluated public baselines. Selecting from eight candidates (best-of-8), YuE2 reaches 6.96, the highest observed mean among all evaluated systems. Expert listening further establishes its competitiveness with proprietary song generators, favoring best-of-8 over Suno v4.5 and yielding nearly balanced preferences against Suno v5. To learn this generation process from recordings without aligned scores, we introduce MERT2 and SheetSage2 to supply semantic and symbolic supervision. MERT2 sets a new state of the art in music representation learning, surpassing previous best results on 14 of 15 MARBLE metrics; SheetSage2 leads 12 of 15 benchmark-metric pairs in our lead-sheet transcription comparison. The same checkpoint follows score edits while largely preserving unedited musical content and generates zero-shot covers without cover-specific training. Its readable score also enables agentic music editing, with external language models translating user feedback into revisions of the composition.