Yue

YuE:

Open Music Foundation Models for Full-Song Generation

GitHub

Abstract

We tackle the task of generating whole-song music audio from given lyrics, dubbed lyrics2song. While text-conditioned music generation models have produced high-quality results on short clips of non-vocal music, generating minutes-long full songs with both vocal and accompaniment parts remains a challenging problem, and we only see some satisfactory results from several close-sourced commercial systems. The challenge of lyrics2song mainly lies in 1) the long context nature of music 2) the complexity of the music signal compared to others (speech, audio effects) 3) the distorted linguistic content and 4) lack of parallel data (lyrics-audio pairs). In this paper, we propose YuE, a series of open foundation language models for lyrics2song, incorporated into the llama family. We demonstrate that our method can model up to 5 minutes length of music audio, follow lyrics condition throughout the whole song, maintain coherent musical structure, generate catchy vocal melodies and appropriate accompaniment. We develop several techniques to achieve this: 1) we apply a semantically enhanced audio tokenizer to reduce the training cost and accelerate the convergence 2) we propose a dual-token technique to enable track-synced vocal-instrumental modeling without modifying the llama decoder-only architecture, enjoying the established infrastructure for scaling and serving 3) we introduce lyrics-chain-of-thoughts to allow the model progressively generate the whole song in a single context following lyrics condition 4) a 3-stage training scheme is proposed to ensure better scalability, musicality, and lyrics controllability.

YuE Model Checkpoints HF-Link

S1-Model COT ICL
English Link Link
Chinese Link Link
Japanese & Korean Link Link
S2-Model Link
Upsampler Link

Examples

Intro: The Model Song

YuE (乐) means "music" and "happiness" in Chinese. For those who find words starting with Yu difficult to pronounce, it can be pronounced as "yeah."

Modeling Diverse Genres & Vocal Styles

Note:

  • - Lyrics are GPT-generated
  • - Future work will include more world music samples, e.g. Beijing Opera

Metal: Step Back

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- The riff is sick.

- I am not an expert in metal music. Maybe this vocal technique is called vocal fry?

- Lyrics following looks good.

Generate music from the given lyrics segment by segment. [Genre] Bass Metalcore Thrash Metal Furious bright vocal male Angry aggressive vocal Guitar [verse] Step back cause I'll ignite Won't quit without a fight No escape, gear up, it's a fierce fight Brace up, raise your hands up and light Fear the might. Step back cause I'll ignite Won't back down without a fight It keeps going and going, the heat is on. [chorus] Hot flame. Hot flame. Still here, still holding aim I don't care if I'm bright or dim: nah. I've made it clear, I'll make it again All I want is my crew and my gain. I'm feeling wild, got a bit of rebel style. Locked inside my mind, hot flame. [outro] Hot blaze. Hot blaze. Still here, still holding aim

Jazz: Quiet Evening

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] female blues airy vocal bright vocal piano sad romantic guitar jazz [verse] In the quiet of the evening, shadows start to fall Whispers of the night wind echo through the hall Lost within the silence, I hear your gentle voice Guiding me back homeward, making my heart rejoice [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [verse] Moonlight paints a picture upon your lovely face Every glance between us fills the empty space Time stands still around us when you're in my arms Nothing else can matter, safe from any harm [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [bridge] Every touch ignites a fire, burning deep within Every smile you give to me makes my head spin Promise me you'll stay awhile, don't ever say goodbye Together we'll chase every star across the sky [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [outro] Stay with me forever, let our love just flow

Rap: This is My Life

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] rap piano street tough piercing vocal hip-hop synthesizer clear vocal male [verse] Woke up in the morning, sun is shining bright Chasing all my dreams, gotta get my mind right City lights are fading, but my vision's clear Got my team beside me, no room for fear Walking through the streets, beats inside my head Every step I take, closer to the bread People passing by, they don't understand Building up my future with my own two hands [chorus] This is my life, and I'm aiming for the top Never gonna quit, no, I'm never gonna stop Through the highs and lows, I'mma keep it real Living out my dreams with this mic and a deal [verse] Late nights grinding, writing down these rhymes Clock is ticking fast, can't afford to waste time Haters gonna hate, but I brush it off Turn the negativity into something strong Mama working hard, wanna make her proud Echoes of her prayers cutting through the crowd Friends turned strangers, but it's all good Focused on my path like I always knew I would [chorus] This is my journey, and I'm running this race Heart full of fire, you can see it in my face Obstacles ahead, but I got no fear Victory is close, yeah, it's almost here [bridge] They said I couldn't do it, said I'd never rise But now I'm soaring high, reaching for the skies Lessons that I learned made me who I am Standing tall now, I don't give a damn [verse] Echoes in the alley, music's getting loud Feeling the adrenaline pumping through the crowd Spotlights on me, it's my time to shine Living in the moment, everything's aligned Looking back now at the roads I've crossed Every single battle, every line I've tossed Made me stronger, wiser, ready for what's next Writing my own story, turning pages of the text [chorus] This is my song, and I'm singing it proud Voices united, hear us shout out loud From the underground straight into the stars Carving out my name, leaving all these scars [outro] Yeah, this is for the dreamers, the ones who never quit Keep your head up high, and don't you ever submit Life is what you make it, so make it something great Step into your purpose, go and seize your fate

Pop: I Won't Back Down

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- Same lyrics but different tags (1-pop/2-pop/3-no pop)

Generate music from the given lyrics segment by segment. [Genre] inspiring female uplifting pop airy vocal electronic bright vocal vocal [verse] Staring at the sunset, colors paint the sky Thoughts of you keep swirling, can't deny I know I let you down, I made mistakes But I'm here to mend the heart I didn't break [chorus] Every road you take, I'll be one step behind Every dream you chase, I'm reaching for the light You can't fight this feeling now I won't back down I'm the whisper in the wind, the shadow by your side The warmth you feel within when you can't hide You know you can't deny it now I won't back down [verse] They might say I'm foolish, chasing after you But they don't feel this love the way we do My heart beats only for you, can't you see? I won't let you slip away from me [chorus] Every road you take, I'll be one step behind Every dream you chase, I'm reaching for the light You can't fight this feeling now I won't back down I'm the whisper in the wind, the shadow by your side The warmth you feel within when you can't hide You know you can't deny it now I won't back down [bridge] No, I won't back down, won't turn around Until you're back where you belong I'll cross the oceans wide, stand by your side Together we are strong [outro] Every road you take, I'll be one step behind Every dream you chase, love's the tie that binds You can't fight this feeling now I won't back down

Ballad: Hospital

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] Sad varied vocal male Country Folk full vocal rich vocal Piano Serious [verse] She arrives here each day This small lady with hair of grey And she merely grins and stumbles On her way [verse] In these hospital hallways She strolls along these silent ways To a chair Where an old woman used to stay And an old woman used to pray I'm in the autumn of my days [verse] I've known the gladness I've known the stress I've just gone through the doctor's mess Please don't sigh, my dear love mine [verse] He arrived here not long past And the physicians were in doubt If the moment for him to pass Was nigh [verse] But when the time arrived And he closed his eyes to rest She pleaded to the Lord above His spirit to bless And as she turned away to grieve It felt like she could sense I'm in the dawn of a new life [verse] I've known the gladness I've known the stress I've just faced the doctor's test Please don't sigh, my dear love best [verse] She'll stay there for quite a while And recall their love with a smile And set it there, the freshest flowers in style [verse] She stands near that silent door Her tired heart is sore and worn And the tales that they once told Still ring in her ears so cold [chorus] I'm in the autumn of my life I've known the joy I've known the strife I've just come out of the sickroom light Please don't sigh, my dear delight 'Cause you and I, we've shared some light [chorus] And our love has reached far and wide And in the glow of stars at night You and I will always unite [end]

Mandarin Pop: My Love

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] Pop vibrant vocal Piano R&B Uplifting Drums Guitar male full vocal Inspiring Mandarin [verse] Hey 你此刻是否还好呀 狂风 在窗外肆意地刮 想起 一同漫步的那一下 你讲会伴我到天涯 我们牢牢牵手不撒 无需再畏惧害怕 [verse] Oh Dear 笑容是否还灿烂 天真 是否已经消散 季节更替了 需添衣保暖 注意别再受风寒 毕竟如今我已不在 你身旁作伴 [chorus] Someday 他给予的誓言已消散 若你回忆那昨天 请你能够念我 My Love [inst] 你我心并未走散 [verse] Oh Dear 笑容是否还灿烂 天真 是否已经消散 气温变低了 要注意保暖 别让自己受风寒 毕竟如今我已不在 陪在你的那一端 [chorus] Someday 那约定的美好未抵达 倘若你追忆往昔 请你试着念我 My Love 我与你并非天涯 [chorus] Someday 他给予的誓言已消散 倘若你回忆过往 请你试着念我 My Heart [outro] 你我心从未走远

Pop: Quiet Evening

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] female airy vocal bright vocal piano sad romantic guitar pop [verse] In the quiet of the evening, shadows start to fall Whispers of the night wind echo through the hall Lost within the silence, I hear your gentle voice Guiding me back homeward, making my heart rejoice [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [verse] Moonlight paints a picture upon your lovely face Every glance between us fills the empty space Time stands still around us when you're in my arms Nothing else can matter, safe from any harm [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [bridge] Every touch ignites a fire, burning deep within Every smile you give to me makes my head spin Promise me you'll stay awhile, don't ever say goodbye Together we'll chase every star across the sky [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [outro] Stay with me forever, let our love just flow

Soul: Hold You Anyhow

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] female romantic keyboard soul emotional bright vocal airy vocal blues classic rock guitar bass drums [verse] I've got to hold you, anyhow Not later, this hour [verse] I know it's dark, whoa I can't mark So come on and take my hand Please, take my hand Now, don't start fretting, trying to understand [chorus] The stars are shining, let's not miss this prime I know it's night, whoa I feel the hype So come on and take the flight Please, take the flight [verse] I know it's hard making this request There's no other path I can take to be your quest [chorus] If only your kin would consent Situations like this we could prevent No, no I won't reveal to anyone I'll hold it within [chorus] I know it's dark, whoa I can't mark So come on and take my heart Please, take my heart [outro] 'Cause I desire you, truly I do Oh, stay by my side Yes, oh, darling I require you I want you, I want you to light my way

Country: Lonesome Road

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] male mellow vocal Guitar Piano Country dark vocal Blues introspective Sad [verse] Da-dum, da-da-da-da-da-da-da Da-dum, da-dum Walking down this lonesome road Thinking 'bout the love untold Why haven't I told you I've whispered to the midnight stars [chorus] Friends keep asking if I'm fine I just smile and say you're mine Maybe you feel the same way too Oh darling, if you do Why haven't you told me Da-dum, da-da-da-da-da-da-da [verse] I've sung it to the morning sun That with you, my life's begun Why haven't I told you My heart's an open book today Waiting for the words to say Why haven't I told you [chorus] Friends keep asking what's the news I just grin and think of you Time to take a chance Let my feelings show Maybe you feel the same way too Oh darling, if you do Why haven't you told me [bridge] Da-dum, da-da-da-da-da-da-da Da-dum, da-dum No more holding back these words Let them fly just like the birds [chorus] Now I'm standing here tonight Hoping that I got it right Maybe you feel the same way too Oh darling, if you do Let's not waste another day Why haven't we told us [outro] Da-dum, da-da-da-da-da-da-da Da-dum, da-dum Now we've finally told us Our new life's begun

Alternative Rock: Corner

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] male emotional alternative rock dark vocal introspective acoustic guitar electric guitar indie rock drums [verse] Standing on the corner, shadows in the night The city's heartbeat echoes under lights Hands deep in pockets, wandering alone Footsteps tracing paths to the unknown Suddenly he pauses, looks up to the sky [chorus] Lost within the chaos, seeking out a sign In a world of color, drawing blurred lines Moving forward, looking back, unsure of the way Trying to find a place where he can stay [verse] He crosses empty streets, under neon glow Faces in the crowd, stories left untold Raises up his arms, reaching for the truth Grasping at the fragments of his youth [chorus] Lost within the chaos, seeking out a sign In a world of color, drawing blurred lines Moving forward, looking back, unsure of the way Trying to find a place where he can stay [bridge] Doesn't want to leave, doesn't want to fight Caught between the darkness and the light No need for reason, nothing to prove Just a soul in transit, with nothing to lose [outro] Doesn't want to leave, doesn't want to fight Chasing after shadows in the night He doesn't need the truth, doesn't need a name Just looking for a spark to fan the flame

Indie: Act

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] indie rock guitar electric guitar full vocal mellow vocal romantic dream pop emotional keyboard drums [verse] Three more days pass and no more heartache These are the thoughts I make I've been searching through the dark Though my spirit starts to break That I could be brave The South would be so glad of me [verse] How would you act [verse] When the moon hides behind the cloud And you're only two steps from breakdown? [chorus] Lemonade, Madeira, heart in disguise So we take our shiny stars And use them as scars Now there's nothing left for me to fear My own tears and silent cheer But I'll stand here The South would be so glad of me [inst] [verse] So I'll meet you the next day Shall we say it's done? Trying to make flames ignite Seeing as they've run You don't know how to cherish that But I know how to cherish that I believe we'll be okay The North would be so glad for you [chorus] What would you say When moons fade in the night And you're just one breath from breaking? [outro] [done]

Childrens' Song

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

Generate music from the given lyrics segment by segment. [Genre] 儿童 female clear vocal 动画 high-pitched vocal 声乐 有趣 打击乐器 欢乐 [verse] [verse] 疙疙瘩瘩,松松垮垮, 歪歪扭扭,坑坑洼洼。 大脑,今天再次启动咯! 行嘞~! 我是疙疙瘩瘩的大脑, 时而迷糊时而明晰。 我是疙疙瘩瘩的大脑, 哟呵!搞清楚了! 我是松松垮垮的大脑, 有时愉快有时烦闷。 我是松松垮垮的大脑, 哎~实在太烦啦! 疙疙瘩瘩,松松垮垮, 歪歪扭扭,坑坑洼洼。 哟~真不轻松呀! [chorus] 我是晃晃的大脑, 思一思后做判定。 我是晃晃的大脑, 嘿!就选定! 我是凸凹的大脑, 壮壮身躯随我令。 我是凸凹的大脑, 哟!要警醒! 皱皱巴巴,软软绵绵, 弯弯曲曲,凹凸不平。 大脑,今天也多谢你啦! [end] 没关系!

Modeling Diverse Languages

Note:

  • - Lyrics are picked from authors' playlists, and rewritten by GPT.
  • - We actually support a large spectrum of languages, both western and eastern. We currently provide demos in English, Chinese (Mandarin and Cantonese), Japannese and Korean.
  • - The model supports code-switching between languages.
  • - Will work on more samples from different languages. Musicality may vary.

Mandarin-English Hiphop: 酷佬

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- Chinese gangsta rap

Generate music from the given lyrics segment by segment. [Genre] hiphop synthesizer tough male rap street bass vocal piano [intro] 哥俩妙 别整那虚一套 都是敞亮人 兄弟你别闹 哥俩妙 别整那虚一套 你心里面藏招 他兜里揣着炮 [chorus] 人称酷佬 酷佬 酷佬 酷佬 街头酷佬 想要roll roll roll roll 富贵财宝 人称酷佬 酷佬 酷佬 酷佬 街头酷佬 想要roll roll roll roll 富贵财宝 [chorus] 都唤他江湖酷佬 ei 江湖酷佬 他渴望荣华财宝 ei 荣华财宝 世界能够那般广 世界亦那般渺 哎哟 你也听闻他 ayo 如此凑巧 [verse] 大伙都在讲你很牛啦 你本事确实大啦 我已然被你折服啦 O~ 你跟那国外嘻哈咖挺像啦 光彩那是相当亮啦 胜过头顶的月亮啦 他的电话声响起ring ring 几句话里几句bling bling 零用钱换了ring ring 说再会 他讲pingping wu 他特豪爽 从不拖拉 能露脸的事儿绝不犯傻 戴着帽子 让你心神不宁 他一现身就好似寒风刮来 他知晓努力就有回报 却不明晓宽以待人 也不在意 渴望迅速拼搏一番 把你当工具使唤 走着山路 un [verse] 他方言说得溜 遇人就喊好兄弟 撸起胳膊 欲做那把头 可他却偏爱那点小便宜 使他名声扫地 把他当挚友 可他把哥们当小弟 对他讲真心话他不珍惜 情谊渐渐地被他遗弃 [verse] 各类事情都想干 这兄弟特别悍 心中偶像周润发 他要做个街头霸 敢冲敢闯 你的善意他当是懦弱 引发争执与怒火 一回回把他当友可这般下去会走散 说对了你别夸俺 For free (这一条不收费哦) [chorus] 人称酷佬 cool佬 cool佬 cool佬 江湖酷佬 想要roll roll roll roll 富贵财宝 人称酷佬 cool佬 cool佬 cool佬 江湖酷佬 想要roll roll roll roll 富贵财宝 都叫他江湖酷佬 ei 江湖酷佬 他想要富贵财宝 ei 富贵财宝 世界能够那么广 世界没有那么窄 哎哟 你也听闻他 ayo 如此凑巧 [verse] 哥们口气放得狂 自称街头王中王 金表得有十来块 神采飞扬够闪亮 场面一定得撑住 潇洒自如压个场 出门开辆法拉利 誓要成为强中强 [verse] 他事先对你一顿呛 习惯事后再给你点希望 不知不觉信任快耗光 临时补救是他的惯样 他仍然能四处闯荡 可情谊却难以再往 忧愁显在他的面庞 深夜的巷子里灯光下像只孤独的野狼 [verse] 渐渐大伙不捧场 他不明所以不转向 他常言未来路还长 但当下他的路摇晃 他的心境渐失常 独自之时难安详 他怨友朋 太吝啬太薄凉 却未看清己模样 并无深仇大恨藏 可为何又遇这状况 一笑能否解怨怅 ask me MR 江湖郎 望着家中财与宝 他不知如何是好 时而欢乐时而忧 做人实在太难熬 哎 这江湖郎 [outro] 哥俩妙 别总瞎胡闹 都是精明人 兄弟你别闹 哥俩妙 别总瞎胡闹

English + Japanese + Korean Code Switching Kpop: 完璧な関係

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- The arrangement of this sample is amazing.

- OG song: https://www.youtube.com/watch?v=v1NMaIQ58N0 OFFICIAL by EXID

[Genre] bright vocal R&B Love airy vocal K-pop Dance Synthesizer Keyboard female Piano [intro] [verse] 完璧な関係 端から見ればね 甘い棘が刺さったままの pain You're not the only one I blame I know I'm not the only one I know you got another one (uh huh) 愛の魔法に溺れるわ Abracadabra... in love with you [verse] 絡まる指先 (oh yeah yeah) Do you want her do you want me (oh no no) 言えない wanna be your number one (ねぇ baby) Come a little closer (hush hush) [chorus] Just take my hand boy Will you be mine このまま この空を追い越して 連れ去って into your world 今すぐに Just wanna be Wanna be your [chorus] Just take my hand boy Will you be mine ここからこの夜を追い越して 連れ去って into your world 永遠に Just wanna be Wanna be your [verse] Official, official, official, official, official, official girl Can we be official, official, Official, official, official, official, boy? [verse] 曖昧なままで また欲しくなって 甘い蜜が降り注ぐ love Rain Oh Baby give me one more time [chorus] Let me be ur official or 있게 해줘 옆에라도 I want you to love me want you to love me ya ya You and I 우리 둘 다 got this feeling babe Want you to want me so tell me honestly [bridge] 그냥 난 원해 너의 9 to 5 I'm on it All day all night long 다 보내 we burn it We got it we want it 네 맘속에 있는거 다 꺼내 우리 둘이 수리 수리 마수리 like magic ya Go tell her about us now And 떠나자 당장 우리 tonight [bridge] ほつれていく指先 (oh yeah yeah) Do you want her do you want me (oh no no) 違うわ wanna be your only one (ねえ baby) I don't wanna hear no... (hush hush) [chorus] Just take my hand boy Will you be mine このままこの空を追い越して 連れ去って into your world 今すぐに Just wanna be Wanna be your Just take my hand boy Will you be mine ここからこの夜を追い越して 連れ去って into your world 永遠に Just wanna be [chorus] Wanna be your Official, official, official, official, official, official girl Can we be official, official, Official, official, official, official, boy? [end]

Cantonese Ballad: 爱你无需讲道理

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

[Genre] female Cantonese Melancholic Classical airy vocal Piano bright vocal Pop Nostalgic Violin [verse] 我也并非无畏惧 我也并非不知险 然而在深情相拥之际 哪怕艰难 崇山峻岭 为你也愿作坦途 爱你无需讲道理 [verse] 但愿用真心实意去陪伴你 一想到心中的你 从未有过的勇气 猛然充满渐乏的身体 [chorus] 旁人全都阻拦 连道理也难容 仍全心奔赴 痛也不会哼 似追逐那片虹 谁人怎样猛冲 都难比我为你这般疯 [chorus] 前路荆棘再多 无人可阻我步 看着是千难万险都敢赴 我不够成熟 仅存有这份顽固 [verse] 我也渴望被关怀 但甘心成为卫士来守护你 荣耀你不赐予我 依旧执意走下去 洒脱笑着为你抗风雨 [chorus] 他人始终质疑 连常理也不顾 仍全心付出 苦都不觉苦 仍执着一份情 谁人道我顽固 亦不如我为你那般笃 [chorus] 前方阻碍重重 无人可让我停 看着那艰难险阻都前行 我并非无情 仅存这腔赤诚 [chorus] 摔下去再站起 就像是小强兵 明明已受挫折 仍努力去前行 [chorus] 他人从不认可 连道理也不通 仍全心付出爱都不觉浓 仍执着一段情 谁人怎样冲动 亦不如我为你那般疯 [chorus] 沿途风雨再猛 无人可阻我行 望着是千难万险都敢迎 哪怕无陪伴 依旧存这份坚定 期待情的人 全部情得很坚定 [bridge] [verse]

Mandarin Rock: 你要跳舞吗

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- We actually did in-context-learning here, using part of the original song as prompt. You can hear the vocalist sounds similar.

- OG song link: https://www.youtube.com/watch?v=4ZaHdnfI6iQ 《你要跳舞吗》by 新裤子乐队

- The generated song is more like a western hardrock. No direct copying here.

Generate music from the given lyrics segment by segment. [Genre] full vocal mellow vocal male electric guitar rock energetic Mandarin rebellious drums clear vocal range vocal [verse] 你要跳舞吗 [verse] 每当潮水涌动之际 你是否也感到心碎 在拥挤而孤寂的房间里 我已无法呼吸 在这冷酷无情的都市中 于时尚颓废的聚会里 [chorus] 你你你要跳舞吗 你你你要跳舞吗 你你你要跳舞吗 你你你你要跳舞吗 你你你要跳舞吗 [verse] 每当潮水涌动之际 我自然也会感到悲伤 在拥挤而孤寂的房间里 我已无法呼吸 在这冷酷无情的都市中 于时尚颓废的聚会里 [chorus] 你你你要跳舞吗 你你你要跳舞吗 你你你要跳舞吗 你你你要跳舞吗 你你你你要跳舞吗 I know y' all was waiting for this flow to come (我知道你们一直期待我的节奏到来) [verse] 我们都是第一,对吧 (We all number one right) 这就是我的感受 (that's just how I feel) 那么,谁想跳舞? (So who wants to dance) 我们想要跳舞 (We wanna dance) [bridge] 我们渴望舞动 [chorus] 你你你你要跳舞吗 你你你你要跳舞吗 你你你你要跳舞吗 你你你你要跳舞吗 [end] 你你你你要跳舞吗

Emergent Spontaneous Performance & Advanced Vocal Techniques

Note:

  • - Lyrics are rewritten by GPT and directly generated by GPT.
  • - We are showcasing advanced vocal techniques and spontaneous performances that demand years of professional training and talent, which have been learned by our 7B model.

Scatting

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- Originating in vocal jazz, scat singing or scatting is vocal improvisation with wordless vocables, nonsense syllables or without words at all.

- Scatting starts when running out of lyrics at the end of the song.

- You can jump to 2:20 if you only want to listen to scatting.

- But the vocal performance is beautiful anyways.

Generate music from the given lyrics segment by segment. [Genre] female blues airy vocal bright vocal piano sad romantic guitar jazz [verse] In the quiet of the evening, shadows start to fall Whispers of the night wind echo through the hall Lost within the silence, I hear your gentle voice Guiding me back homeward, making my heart rejoice [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [verse] Moonlight paints a picture upon your lovely face Every glance between us fills the empty space Time stands still around us when you're in my arms Nothing else can matter, safe from any harm [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [bridge] Every touch ignites a fire, burning deep within Every smile you give to me makes my head spin Promise me you'll stay awhile, don't ever say goodbye Together we'll chase every star across the sky [chorus] Don't let this moment fade, hold me close tonight With you here beside me, everything's alright Can't imagine life alone, don't want to let you go Stay with me forever, let our love just flow [outro]

Death Growl

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- It is pretty loud. Turn down your headphone!!

- You can start at 0:50.

- The lyrics following is quite well even under this extreme low vocal-to-accompaniment ratio.

- Solo is a bit long and repetitive.

- Actually, we support chinese genre tags to some extent.

Generate music from the given lyrics segment by segment. [Genre] 鞭笞金属 powerful vocal male 吉他 死亡金属 残酷 贝斯 boomy vocal 暴力 [solo] [solo] [verse] I, the Moon of night The child of the cosmic sight My shine dimmed and lost within the sky That I may find harmony in this land [verse] I, child of rebellion From pure emptiness defied Towards the loftiest soul, to boundless liberty To uncover the fiery essence of my plea [solo] [verse] I, spark of creation The truth of nature I embrace I carry the flame of Prometheus So I can light the dark and guide the stars [chorus] I, celestial rebel Infusing frenzy in my soul With hope embraced, with tears unleased Is the wisdom of gods in man whole? [solo] [solo] [verse] The light of the morn The thrill of the eve Sustained am I by this cosmic wave Where high above blends with deep below [verse] Let it be spoken, let it be known Wandering I march towards the broken throne [solo] [solo] [solo] [solo] [solo]

Mix Voice

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- Mix voice is the blending of the chest voice and the head voice. It is also the elimination of the bridge- that pesky gap that connects the two registers.

- Commonly seem in power metal.

Generate music from the given lyrics segment by segment. [Genre] Power metal Guitar male Energetic bright vocal airy vocal [intro] [intro] [intro] [verse] Moments of love Moments of pain Floating by within my brain Now, then, soon, a single chain Nights of fun Nights of woe Come and leave as I still know A week, a month, a thousand days, they go Oh, someday I will depart to find a new way [chorus] No, and this world won't pause to spin along with me 'Cause time Marks time Marks on without us free Ever goes [verse] Sure time Speeds time Speeds along and never gone Flows continuously [verse] Times of hope Times of strife Endless journey is our life Can't give up now, not till we're through We yearn for more serenity, and perhaps a new life This one is brief, no matter what you do [chorus] Oh, don't ever lose the faith to live a great life No, don't squander your precious time to make it strife [chorus] Time Flows time Flows on leaving us behind Never pauses Yes time Moves time [chorus] Moves along and never gone Flows continuously [solo] [beat] [beat] [chorus] Please please let me find The path that's truly mine Make a move and we shine so fine [chorus] No more idle days No more futile ways Life's too brief to sigh Long enough to fly [chorus] Time Flees time Flees away without a sound Never slows [chorus] Yes time Moves time Moves along and never gone Flows continuously [new_end]

Powerful Belt, Riffs and Runs 01

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- This sample is from an old checkpoint.

- Tons of riffs and runs. But less control.

Generate music from the given lyrics segment by segment. [Genre] pop In the heart of knowledge, where dreams take flight, A vision unfolds, bathed in radiant light. The AMC stands proud, a beacon so bright, Where art meets machine, a future in sight. With creativity sparked, and minds set ablaze, Innovation will flourish in myriad ways. From pixels to patterns, in harmony blend, A journey begins, where boundaries extend. Artists and thinkers, together they'll weave, A tapestry rich, in which all can believe. With each brushstroke and code, a story will grow, In this realm of wonder, new ideas will flow. Let's celebrate this moment, this path we embrace, In the dance of the digital, we'll find our place. At HKUST, where the future ignites, The Arts and Machine, a symphony of sights. So here's to the dreamers, the makers, the bold, In this vibrant new division, let visions unfold. Together we’ll craft, innovate, and inspire. At AMC, we’ll reach ever higher! [outro]

Powerful Belt, Riffs and Runs 02

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- This one is new.

- Sounds great. But a lot of distortion at high vocal range. We plan to address it in the next version.

Generate music from the given lyrics segment by segment. [Genre] Keyboard Uplifting Piano Pop airy vocal bright vocal Electro pop Hopeful female Synthesizer [verse] Was 25 striving to find my path to Neptune A short escape didn't seem so wrong Was filled with doubts tied up hard like concrete Running on low, so nearly gone Thought I wasn't wise And I wasn't so strong [verse] Sitting in the darkened room We were tied with endless gloom And I thought the blame was mine Couldn't bear it anymore [chorus] With the strength from within (No more hiding in the shade) I lifted my spirit high (I made a choice to fly) I took one step after another and I [chorus] Faced the mirror and chose to remain Wasn't gonna let love lead me astray This way [verse] I thank my brother for keeping my spirit from drowning When the reality was like chewing on lead Now every dawn There is no more sadness, oh yeah I can truly find myself anew I know I am worthy Able to be cared It wasn't for me to quit Now I must soar beyond Let the cosmos prove it wrong Yeah the truth will make me strong [chorus] By the strength of soul (There was no turning back) I raised myself anew (I knew I couldn't slack) I took one step after another and I [chorus] Stared at the mirror and chose to remain Wasn't gonna let heartache drive me insane That way no There ain't no Not in the name of pain In the name of pain (In the name of pain) That way There ain't no I'm not giving in [chorus] By the strength of will I made myself stand tall I take one step after another and I Looked to the future (Looked to the future) Looked to the future (Looked to the future) [chorus] With the strength from within (No other path to win) I lifted myself once more (I realized I must endure) I take one step after another and I [outro] Stared at the mirror and chose to remain Wasn't gonna let heartbreak drive me insane This way [outro]

A cappella

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- We do not support the tag "a cappella". But this sample is prompt with in-context-learning.

- Lyrics are rewritten by GPT.

- OG song: https://www.youtube.com/watch?app=desktop&v=Urf7wDavQKw&t=0s

Generate music from the given lyrics segment by segment. [Genre] happy clear vocal full vocal uplifting hip-hop rap range vocal [verse] How does a waif, an outcast, child of a strumpet and a Welshman, placed in the midst of an overlooked Nook in the Atlantic by Destiny, destitute, in disarray Rise up to be a legend and a sage Well the news spread wide, they claimed, this lad is remarkable, man Raised a fund just to dispatch him to the mainland Gain your knowledge, don't neglect where you originated, and The world will hear your fame, what's your name, lad Alexander Hamilton (Alexander Hamilton) We are standing by for you (standing by for you) You could never give in You never learned to slow your spin [chorus] Oh, Benjamin Franklin (Benjamin Franklin) America praises you Will they know what you achieved? Will they know you changed the scheme? The world will always remember, Benjamin [verse] You'll come back In no time You'll recall that you're mine You'll come back Fate will shine You'll recall that I'm fine [verse] Mountains stand, kingdoms crash We've journeyed together, through every clash And when times get tough, I'll dispatch a force that's truly enough to show you my love's vast [chorus] Ba baba ba ba Ba bababa bayaba Baba ba ba bayaba Ba baba ba ba Ba bababa bayaba Baba ba ba ba I won't let my chance go past I won't let my chance go past Hey yo, I'm as strong as my land I'm brave, eager, and grand And I won't let my chance go past [verse] I'ma gain entry to King's College fair I perhaps shouldn't boast, but eh, I shock and astound The hitch is I've got much wit, but lack grace I must shout just to be noticed With each word I impart wisdom I know the scene on the street is thrilling But Lord, amid all the bleeding and brawling I've been studying and penning We ought to address our fiscal condition Are we a land of states, what's the state of our land I'm beyond patient waiting! I'm fervently shattering every anticipation Every deed is an act of innovation I'm smiling in the face of losses and grief For the first time, I'm looking beyond today [chorus] And I won't let my chance pass by I won't let my chance pass by Hey, yo, I'm just like my land I'm brave, strong, and ready to stand And I won't let my chance pass by I won't let my chance pass by I won't let my chance pass by Time to make a try Make a try And I won't let my chance fly [bridge] Not giving up my quest You will thrive with our newfound land We'll strive and toil for you We'll make it fine for you If we build a sturdy base We'll hand it down to you We'll offer the world to you and you'll astound us all through Someday, someday Yeah you'll astound us all through [chorus] I long to be in The place where dreams take flight The place where dreams take flight The place where dreams take flight I long to be in The place where hopes ignite The place where hopes ignite The place where hopes ignite No one truly grasps how the path is laid The skill of the chase How the stars are made We merely suppose that it's made But no one else can take this place [bridge] The place where it starts [chorus] I kept every gift you gave to me From the instant I received them I felt you were mine You claimed you were mine I believed you were mine [outro] Oh, I long to hold you once more It's merely a question of Time Will they recount your tale Time Who thrives, who fades, who narrates your story Time Will they speak of your glory Time [outro] Who weeps, who hopes, who shares your story Who dreams, who schemes, who cares your story

Harmonica solo improvisation

⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

- A harmonica improvisation at 2:00.

Generate music from the given lyrics segment by segment. [Genre] Folk Pop male dark vocal Inspiring Piano Uplifting Guitar mellow vocal [verse] 斑马斑马 别再沉睡了 让我看见你伤痕累累的尾巴 我不愿触碰你那道疤痕 只想轻拂你飘逸的发丝 [verse] 斑马斑马 你已归于故乡 而我却在寒冷中虚度年华 你的城市对我关闭了所有的门扉 终究我只能再次踏上孤独的旅途 [chorus] 斑马斑马 你来自南方的绯红 这是否也是一段动人心弦的故事 若你身旁的戏子无法留下 谁会陪伴我直到天明 [chorus] 斑马斑马 你还记得我吗 我是那个只会歌唱的傻瓜 [chorus] 斑马斑马 安眠吧安眠吧 我将背起吉他离开北方 [verse] 斑马斑马 你还记得我吗 我是那个装作忧愁的孩子 [chorus] 斑马斑马 安眠吧安眠吧 我会将你的青草带回故乡 斑马斑马 你还记得我吗 [chorus] 我只是一个匆忙的旅人啊 斑马斑马 安眠吧安眠吧 我要卖掉我的房子 漫游于天涯 [outro]

Thanks to the following organizations for their support

图片1 图片2 图片3

And also thanks geely

Citation

@misc{yuan2025yue,
    title={YuE: Open Music Foundation Models for Full-Song Generation},
    author={Ruibin Yuan and Hanfeng Lin and Shawn Guo and Ge Zhang and Jiahao Pan and Yongyi Zang and Haohe Liu and Xingjian Du and Xeron Du and Zhen Ye and Tianyu Zheng and Yinghao Ma and Minghao Liu and Lijun Yu and Zeyue Tian and Ziya Zhou and Liumeng Xue and Xingwei Qu and Yizhi Li and Tianhao Shen and Ziyang Ma and Shangda Wu and Jun Zhan and Chunhui Wang and Yatian Wang and Xiaohuan Zhou and Xiaowei Chi and Xinyue Zhang and Zhenzhu Yang and Yiming Liang and Xiangzhou Wang and Shansong Liu and Lingrui Mei and Peng Li and Yong Chen and Chenghua Lin and Xie Chen and Gus Xia and Zhaoxiang Zhang and Chao Zhang and Wenhu Chen and Xinyu Zhou and Xipeng Qiu and Roger Dannenberg and Jiaheng Liu and Jian Yang and Stephen Huang and Wei Xue and Xu Tan and Yike Guo}, 
    howpublished={\url{https://github.com/multimodal-art-projection/YuE}},
    year={2025},
    note={GitHub repository}
}