In a Universal Music Translation Network, let $$s^j$$ be an input sample from domain $$j = 1, 2, \dots, k$$, where $$k$$ is the number of domains employed during training. Let $$E$$ be the shared encoder, and $$D_j$$ be the WaveNet decoder for domain $$j$$. Let $$C$$ be the domain classification network, and $$O(s, r)$$ be the random augmentation procedure applied to a sample $$s$$ with a random seed $$r$$. The reconstruction error is formulated as:

$$\sum_j \sum_{s^j} E_r L(D^j(E(O(s^j,r))), s^j)$$

University of Michigan - Ann Arbor

Google

It's the sota baseline of music style transfer tasks proposed by groups from facebook, 2018. Because on one hand, this framework could transfer timbre, music style like jazz, traditional music or others, and performance. On the other hand, this framework introduces the domain confusion error combined with reconstruction error, which convinces that the styles do influence the music. As a result, it's still sota today and a milestone work of music style transfer. 

A Universal Music Translation Network

In a Universal Music Translation Network, let $$s^j$$ be an input sample from domain $$j = 1, 2, \dots, k$$, where $$k$$ is the number of domains employed during training. Let $$E$$ be the shared encoder, and $$D_j$$ be the WaveNet decoder for domain $$j$$. Let $$C$$ be the domain classification network, and $$O(s, r)$$ be the random augmentation procedure applied to a sample $$s$$ with a random seed $$r$$. The classification loss is calculated as: $$\sum_j \sum_{s^j} \lambda L(C(E(O(s^j, r))), j)$$.

Learn Before

Related