Let $s^j$ be an input sample from domain j = 1, 2, . . . , k, k being the number of domains employed during training. Let E be the shared encoder, and $D_j$ be the WaveNet decoder for domain j. Let C be the domain classification network, and O(s, r) be the random augmentation procedure applied to a sample s with a random seed r.
The reconstruction error is:
$\sum_j \sum_{s^j} E_r L(D^j(E(O(s^j,r))),s^j)$

Reconstruction Error

Let $s^j$ be an input sample from domain j = 1, 2, . . . , k, k being the number of domains employed during training. Let E be the shared encoder, and $D_j$ be the WaveNet decoder for domain j. Let C be the domain classification network, and O(s, r) be the random augmentation procedure applied to a sample s with a random seed r.
The classification loss is:
$\sum_j \sum_{s^j} \lambda L(C(E(O(s^J,r))),j)$

Classification Loss

It's the sota baseline of music style transfer tasks proposed by groups from facebook, 2018. Because on one hand, this framework could transfer timbre, music style like jazz, traditional music or others, and performance. On the other hand, this framework introduces the domain confusion error combined with reconstruction error, which convinces that the styles do influence the music. As a result, it's still sota today and a milestone work of music style transfer. 

University of Michigan - Ann Arbor

Shortly music style transfer means combining two different styles of music into new one. 

Music Style Transfer 

Style transfer is applying the style, characteristics of new image, video, voice, music for the input. Image Style Transfer was quite successful, nevertheless the same can't be said about other areas. 

What is style transfer ?

1. Score Representation
2. Performance Control Representation
3. Sound Representation

Different Representations of Music

1. Timbre Style Transfer (sound)
2. Performance Style Transfer (control)
3. Composition Style Transfer (score)

Music Style Transfer For Different Music Representations

1. https://arxiv.org/pdf/1803.06841.pdf?fbclid=IwAR2Q2RfvxXAG_G1475NUA9qBssBDSOECQl3-ICohC8gT1dT6j3nz0Q48_7Y

2. https://arxiv.org/pdf/1809.07575.pdf?fbclid=IwAR20iNWH9x_ENRuXxrVsl9YDdQO3i1ien-MsAm5XEXHoL0PVcQVVJMq1oAY

3. https://github.com/imalikshake/StyleNet?fbclid=IwAR3coA5NyXC2CaLiEt-twhy3ULu3B-6A5XLOzqDshO1RKvJByrN5uoYeWpE

4. https://arxiv.org/pdf/1910.03713v2.pdf?fbclid=IwAR0lxQhoKhc5N0ipUfDI_n0otE3lRtDRJgBot5mQDQBXG6RNAx7QFxb3W3M

5. https://arxiv.org/pdf/1708.03535.pdf?fbclid=IwAR2_fqrl6Byuh9K4tVdZo7bzGpcwMD674iGVAdvOnGKg1AOCUhLw6fMfYAQ

6. https://lib.ugent.be/fulltxt/RUG01/002/785/848/RUG01-002785848_2019_0001_AC.pdf?fbclid=IwAR2VLS9RspkGTH-8H9YmvDrbQ1AqRMKwzLNeaTcMNE6f4UDGCVBaspeNorQ

7. https://software.intel.com/content/www/us/en/develop/articles/neural-style-transfer-on-audio-signals.html?fbclid=IwAR05mMmyqwKUBLOMH7kBfjk2TrtDdHA86capsEwaZb2wV4ssrYJLHexbOpA

8. https://ieeexplore.ieee.org/document/9178446

Useful References for Music Style Transfer:

1. Create covers for songs
2. Bring pleasure to humans 
3. Understand the key characteristics of the music. 
4. Create new music that is both human like and creative.
5. "One can only imagine the amount of effort that goes into arranging an entire symphony based on a comparatively simple rock song. Domain
transfer systems could significantly accelerate this process, or
even automate it completely, which would let us enjoy music
that generally has not been feasible to create on a large scale." 

Purpose 
(Music Style Transfer)

1. The term "music style" is difficult to be defined. 
2. Music has multi-level, multi-modal representation. 
3. Existing studies can still hardly achieve multi-modal and non-deterministic mapping between different domains. However, when we transfer a piano solo piece into guitar solo, we often expect the outcome of the guitar solo to be adjustable, perhaps with various ﬁngering styles, brightness, musical texture, or other sound quality.

Challenges
(Music Style Transfer)

There are three basic music model constructing foundations:
1. Conditions or labels could be added so as to realize explicit control. And we could try to maximize the "mutual information" introduced by infoGAN to train our model.
2. In Deep Variational Model or Seq2seq models, the high-dimensional representation of music and the styles are decoupled, and the styles are replaced with other style representations so as to realize the music style tranfer.
3. It's actually a subset of intuition 2. However, it's widely used in timbre transfer models. As a result, it's separately introduced here. We could transfer our music to pics and use methods based on neural style transfer to realize the music style transfer.

Basic Intuitions of Music Transfer Model

A Universal Music Translation Network

Music Style Transfer: A Position Paper, proposed by Prof. Gus. Xia, etc.
https://arxiv.org/abs/1803.06841

Learn Before

Related

Learn After