Music from MIDI files
First we downloaded all the MIDI formatted songs that we could find on the internet and transposed all of them to C-Major. The recommended software for dealing with MIDI files would be MuseScore, but also Apple's GarageBand can open the files easily. Please be aware that MIDI is an old and ugly format, dealing with it is a bit of a nightmare.
For the whole project we were using Python versions 2.7 and 3.6. The library to manipulate the MIDI input data of our choice was music21. The library is powerful but also buggy and we needed several work-arounds to be able to extract all the tracks we wanted.
As not enough "voice-tracks" could be found, we decided to focus solely on the guitar and bass tracks which we extracted from all the transposed songs and serially concatenated them. After having inspected the music, the songs were quantized in sixteenths (1/16) notes, i.e. the base note became a "1/16-th note". Next to the 6 parallel tracks of the guitar (6 strings) and the one track from bass we added 2 bits for "mute" and 2 bits for "hold". With the "hold" bit, notes that are longer than 1/16-th can be encoded; this is done by multiplying the notes and setting the "hold-bit" to "1".
The numbers encoding the pitch (0..127) of each note were directly taken from the MIDI files.
By chaining all 130 songs we ended up with about 150'000 lines of 1/16-th notes of which about 2'200 were unique combinations of
guitar, bass and mute/ hold bits. These ~2'200 unique combinations were hot-encoded
("bag of words") and used to train a LSTM neural
network. The chosen input length was 32 and 64 respectively, ie. 2 or 4 bars were used as inputs.
The training data (X) was generated by splitting the notes into blocks of either 32 or 64 1/16-th quantized chords (guitar & bass) that had been hot-encoded; the expected output (y) corresponding to the input data (X) used for training the network was the following chord. The output vector (y) was hot-encoded as well.
The networks used were LSTM-RNN's programmed using "Keras on "Tensorflow; sizes varied but the following is one configuration that provided good results:
| The code written in Python with Keras: | |
|---|---|
|
maxlen = 32 #max. seq.-length: 32 or 64 model = Sequential() model.add(LSTM(128, return_sequences=True, input_shape=(maxlen, len(words)))) model.add(Dropout(0.2)) model.add(LSTM(128, return_sequences=False)) model.add(Dropout(0.2)) model.add(Dense(len(words))) model.add(Activation('softmax')) |
|
Figure: The 10 most frequently played Ramones chords (transposed to C-Major)
Implementations and Losses
The following LSTM-RNNs were implemented and trained resulting the described losses:
| Implementations and Losses | ||
|---|---|---|
| Implementation 1: maxlen = 64 step = 3 model = Sequential() model.add(LSTM(128, return_sequences=True, input_shape=(maxlen, len(words)))) model.add(Dropout(0.2)) model.add(LSTM(128, return_sequences=False)) model.add(Dropout(0.2)) model.add(Dense(len(words))) model.add(Activation('softmax')) |
Implementation 2: maxlen = 32 step = 3 model = Sequential() model.add(LSTM(128, return_sequences=True, input_shape=(maxlen, len(words)))) model.add(Dropout(0.2)) model.add(LSTM(128, return_sequences=False)) model.add(Dropout(0.2)) model.add(Dense(len(words))) model.add(Activation('softmax')) |
Implementation 3: maxlen = 32 step = 1 model = Sequential() model.add(LSTM(128, return_sequences=True, input_shape=(maxlen, len(words)))) model.add(Dropout(0.2)) model.add(LSTM(128, return_sequences=False)) model.add(Dropout(0.2)) model.add(Dense(len(words))) model.add(Activation('softmax')) |
|
min. loss: 0.8142 Total params: 1'700'136 |
min. loss: 0.9395 Total params: 1'700'136 |
min. loss: 0.9009 Total params: 1'700'136 |
| Implementation 4: maxlen = 64 step = 1 model = Sequential() model.add(LSTM(128, return_sequences=True, input_shape=(maxlen, len(words)))) model.add(Dropout(0.2)) model.add(LSTM(128, return_sequences=False)) model.add(Dropout(0.2)) model.add(Dense(len(words))) model.add(Activation('softmax')) |
Implementation 5: maxlen = 32 step = 1 model = Sequential() model.add(LSTM(64, return_sequences=True, input_shape=(maxlen, len(words)))) model.add(Dropout(0.2)) model.add(LSTM(64, return_sequences=False)) model.add(Dropout(0.2)) model.add(Dense(len(words))) model.add(Activation('softmax')) |
Implementation 6: maxlen = 32 step = 1 model = Sequential() model.add(LSTM(128, input_shape=(maxlen, len(words)))) model.add(Dropout(0.2)) model.add(Dense(len(words))) model.add(Activation('softmax')) |
|
min. loss: 0.9119 Total params: 1'700'136 |
min. loss: 1.1968 Total params: 802'088 |
min. loss: 0.8712 Total params: 1'568'552 |
For all the implementations the batch size was kept constant at 128. The relations between the minimum loss and the structure of the LSTM-RNN, as well as the trainings-data was not apparent. It can however be seen that the loss is larger when there are less parameters to train (cf., Implementation 5). All RNNs were trained until the loss seemed to have reached (or was stuck stuck in) a (local) minimum. The training data of about 150'000 chord sequences is probably too small for reliable results and more meaningful interpretations.
Trainings Data
After about 50 to 120 epochs of training (depending on the network), the error-rate was not decreasing anymore; this took around 20-50 minutes on a NVIDIA K80 TESLA GPU. The training data could be reduced by increasing the step size - this had a big (linear) impact on the training times, but did not affect the final error much.
No validation data was put aside for verification/ testing.
Not only the weights of the fully-trained networks were stored, but also intermediate results. In some cases, these produced better results; this effect was however not further investigated.
