🇮🇪🎻 IrishGPT

About

IrishGPT is a small supervised learning model (founded on the GPT architecture) trained on a dataset of traditional irish music tunes. The dataset is the aggregated set of tunes available on the plateform TheSession.org.

The model is designed to generate a tune – a unique piece of music in the traditional irish music repertoire – from scratch or from a specific key and rythm. The program works well but due to the limited extent of the dataset and the small size of learning parameters of the chosen architecture, it can sometimes give weird results.

The overall result is, however, quite astonishing as the model is able to respect the AABB form of the tunes, the successive cadences and the correct scale and rythm. It's a student project with the objective to understand transformers in machine learning and the subsequent GPT architecture.

Why irish music ? Why another GPT ?

Irish music for two reasons. First, there's already a quite large dataset available online published by The Session. This plateform gathered more than 20 000 tunes and all of them are written with the ABC system, a score notation based on ASCII symbols. The Session maintener regularly publishes all the tunes in a single csv file.

Second, because irish music tunes are almost everytime monophonic, short and equally structured around the form AABB. It's then appropriate for a machine learning algorithm which must take a rather small input sequence.

The GPT architecture has been chosen because of the form of the input : a sequence of music symbols (in this case made up of ascii caracters). GPT is a auto-regressive transformers decoder perfectly suited for the generation of a new token based on a previous serie of tokens. In our case, it predicts text. Trained on a dataset of ABC notations, the model is able build a new sequence of ABC symboles iteratively.

Eventually, the attention mechanism behind GPT allows the model to understand by itself the specificity of the irish traditional music, specially those of jig and reels : the question-answer form of music sentences (with the appropriate cadences), the composition of two music sentences per tune. Also, attention mechanism offers to the decoder a self-acknowledgment of previous and important informations such as the key or the rythm decided by the user.