MUSIC TRANSFORMER: GENERATING MUSIC WITH LONG-TERM STRUCTURE

The paper "Music Transformer: Generating Music with Long-Term Structure" investigates the use of the Transformer model, enhanced with a novel relative attention mechanism, to generate music with coherent long-term structure. The research aims to address the challenge of modeling music, which requires maintaining coherence over long sequences due to its inherent repetitive structure.

The authors propose a modified relative attention mechanism that reduces memory complexity from quadratic to linear in sequence length, making it feasible to apply to long musical compositions. This approach allows the Transformer to generate minute-long compositions with compelling structure and to generate continuations that elaborate on given motifs. The model is evaluated on the JSB Chorales and Maestro datasets, achieving state-of-the-art results on the latter.

Key results demonstrate that the Transformer with relative attention can generate music with improved sample quality and perplexity compared to existing models. The model captures both local and global timing, resulting in regular phrases and coherent musical structure. Listening tests indicate that samples from the model with relative attention are perceived as more coherent than those from a baseline Transformer model.

The study acknowledges limitations in the original Transformer model's ability to capture periodicity and relational information, which the relative attention mechanism addresses. Future work could explore extending this approach to other domains, such as long text sequences or audio waveforms, where capturing long-term dependencies is crucial.

The paper suggests that the Transformer with relative attention could serve as a creative tool for musicians, enabling interactive music generation. The authors also highlight the potential for further improvements in time-series models by enhancing the Transformer's ability to capture periodicity and relational information.