~ Linux tricks and other computer wizardry.

Moving and transcoding an m3u playlist with a bash script.

The problem here is that you have an m3u playlist with your favourite songs in it. You want to put all the songs on your latest mp3 player. However, some of them are in the wrong formats and they are scattered all over your computer. You need some way of easily collecting them all together in one folder, with the playlist order preserved all in mp3 format.

This guide will explain how to write and use a bash script that helps you to do this. Before starting you need a folder with the m3u playlist in where you want to put all the mp3 files. The m3u format is fairly common. Most music library programs can export to it if it is not already their default playlist format. You will also need FFmpeg installed on your computer.

Starting the script and allowing the computer to find it
All scripts need to start with a line telling the computer what shell to run the script with. For bash scripts this line is:

#!/bin/bash

this is what we use in this guide. You can replace the /bin/bash part with any path to any script interpreter, Python, gnuplot etc.

To run the script you need to make it executable, by changing the file permissions. Run this in your bash shell

chmod +x m3u-script

this will add (+) the execution permission (x) to any user for that script. It is possible to restrict who is allowed to execute the script depending on owner and group, but that is a topic for another time. Google Linux file permissions if you are interested.

Your script must also be somewhere where Linux can find it. Normally this would be in the directory you are currently in, and would be accessed by typing

./m3u-script

On the other hand you may prefer to collect a whole folder for scripts and add that folder to your $PATH variable by adding this line to your .bashrc file.

export PATH=$PATH:/path/to/scripts/folder/

This will allow you to run any executable file in that folder just by typing its name into the command line.

We also need to initialise a variable to store the song number in. This is done with:

count=0

Unlike other programming languages, you do not need to specify a type when you initialize a variable.

Passing the m3u filename as a command line argument
We need some way of telling the script which m3u playlist we would like it to read. The easiest way is to pass the name of the playlist file on the command line when we call the script. If you call the script in the following way

m3u-script my-playlist.m3u

the strings “m3u-script” and “my-playlist.m3u” will be available inside the script as the variables $0 and $1 respectively. We only need to use $1 to open the playlist.

Due to the way ffmpeg uses the standard input and outputs (which we will use later on) we need to read the entire file into an array before we start moving and transcoding the actual songs. This is done with the following lines:

old_IFS=$IFS
IFS=$'\n'
lines=($(cat $1))
IFS=$old_IFS

To explain, IFS is an input system, and so we don’t overwrite it, we store the previous value to a variable, overwrite it, then put the original value back at the end. The main part of this is lines=($(cat $1)) which tells the computer to list the whole file, and store it as a variable. This variable is actually an array as we changed the delimiter to the newline character in the IFS variable. So the variable lines ends up as an array where each element is a single line from the m3u file.

Looping over every line in the file
We need to look at each line individually and decide what to do with it. First we count how many lines there are with

size=${#lines[@]}
echo "size is "$size

this stores the size of the array "lines" in the variable size. It then prints to the terminal with echo the size so the user knows how many lines the script will process. The number of lines is double the number of songs because each song takes two lines.

We then loop over all the lines with a simple for loop:

for ((i=0;i<$size;i+=1))
do
echo "Dealing with line "$i" where line text is "${lines[$i]}
#Body of loop
done

this says start the index variable i at 0, and add one each time you go through the loop, but only loop while i is less than the value of the size variable we just found. This prevents the script trying to read outside the array we just defined. The echo again just lets the user know where the program has got to. The rest of the sections need to go in the body of the above loop.

Ignoring lines that don’t correspond to filenames
We only want to deal with actual filenames, we can ignore the rest of the information about the song that the m3u format provides. There is an easy way to do this, and that is to just ignore any line beginning with “#”. To access the current line we need ${lines[$i]} which gets the current line as an element of the array “lines” and returns the value of it. We then test this line like this:

if [[ ${lines[$i]} == \#* ]]
then
continue
fi

The condition in the if statement is a regular expression, sometimes called regex. To cover regular expressions properly would take too long, so I will just say that this one matches any line that begins with a “#” character. This if statement is very simple, it says if the line begins with a “#”, continue. The continue refers to the for loop containing the if, meaning that the program should continue from the beginning of the next loop and not do anything else this time round.

Extracting the filename
If the line doesn’t begin with a “#” then it is a path to the next song and we need to deal with it. To be able to deal with it properly we need to extract the filename from the path. This is achieved with parameter expansion

filename=${lines[$i]##*\/}

Again to fully go through this would take too long, it is enough to say that this sets the variable filename to be the text that comes after the last “/” character in the string.

We should also tell the user what we are doing:

echo "filename is "$filename

Notice how in a bash script you can just put strings next to each other, and bash will echo them as one string.

Also, now we have identified a file to deal with, we should update the count that keeps track of the song number.

count=$((count+1))
countstr=$(printf "%04d" $count)

This is simple, it evaluates count+1 and stores it back in count. It then creates a 4 digit string version with the printf "%04d" $count command. We will need this string to prepend the song number to the filename.

Dealing with files that are already mp3
If the file is already mp3 all we need to do, is prepend the track number (count) to the filename and move it to the current directory. The way we test for this is with another regular expression.

This regular expression test for the last three characters of the string being mp3. If they are, it just moves the file using cp to "./${countstr}_${filename}". This will put the file in the current directory, and prepend the 4 digit song number with an underscore before the filename. If you want your filenames to have a different form, eg. prepend with a dash, just change this line here (and equivalent lines in the other sections).

Transcoding files from other formats
Similar to the previous section, we test for the type of file. In my script, I have a separate section for each other possible type but as they have the same structure I will use the example of flac. The main difference here is that instead of copying the file and renaming it with cp we need to actually transcode it (change it’s format). This is achieved through the use of FFmpeg which can transcode audio and video amongst other things.

Firstly notice what is similar and different to the mp3 section. The test is the same, just testing for a different ending. Telling the user what is happening, we just need to tell them that we are transcoding it rather than just moving it.

The other part, ffmpeg has quite a lot of arguments to it. The -i "${lines[$i]}" specifies to take the file we are currently looking at as the input. Then -acodec libmp3lame -ab 192k tells ffmpeg to encode to mp3 file using a bitrate of 192k (which is fairly good and to my ears indistinguishable from a CD). The argument without a dash code is the output file which uses parameter expansion again to remove the .flac part, and adds .mp3 instead. Finally the & tells bash to run this process in the background. The next line, wait $!; tells the script to wait until the previous process has completed before moving on. This just makes sure that we don’t open too many ffmpeg processes all at once.

The full script
I have attached the full script to this blog post, licensed under GPLv3, so you can run it and change it as you like. Have a play with changing things and adapt the script to your needs. I hope this post gives you a step up to understanding bash scripting, and probably a useful script.

Here it is:

#!/bin/bash
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/gpl-3.0.html
# for the full license.
# This program was written by Dominic Hosler dom@dominichosler.co.uk
# There is a description of how it works on my blog https://dominichosler.wordpress.com
# Please ask on the blog if you have any questions, thank you.
#setup song count so order is preserved
count=0
#necessary to fill an array first, because ffmpeg messes with the standard input and output.
#It is not the waiting that is the problem, it is ffmpeg using the input and output that removes
# the next so much of the input lines, however if all the input is done into an array before
# ffmpeg is called, it runs perfectly.
#store old IFS so can return it at the end
old_IFS=$IFS
#set IFS to newline
IFS=$'\n'
#store an array filled with each line of the file.
lines=($(cat $1)) # array
#replace the old IFS
IFS=$old_IFS
#find size of file
size=${#lines[@]}
echo "size is "$size
#loop over the number of lines in the array
for ((i=0;i<$size;i+=1))
do
echo "Dealing with line "$i" where line text is "${lines[$i]}
#if the input line begins with a hash (regex test) ignore it
if [[ ${lines[$i]} == \#* ]]
then
continue
#echo "ignoring "+${lines[$i]}
#count=$((count-1))
fi
#extract the file name from the path (parameter expansion)
filename=${lines[$i]##*\/}
echo "filename is "$filename
#increase the count, and convert to a four digit string
count=$((count+1))
countstr=$(printf "%04d" $count)
#if the input line ends in mp3, only need to move it
if [[ ${lines[$i]} == *mp3 ]]
then
echo "moving file "${lines[$i]}
#copy the file to the working directory and rename it to have the count prefixed
cp "${lines[$i]}" "./${countstr}_${filename}"
fi
#if the file is flac, (or ogg later, it's the same)
if [[ ${lines[$i]} == *flac ]]
then
echo "transcoding and moving file "${lines[$i]}
echo "filename is "$filename
#transcode original file, into mp3 192k bitrate, calling the new file, count_filename (without extension) .mp3
ffmpeg -i "${lines[$i]}" -acodec libmp3lame -ab 192k "${countstr}_${filename%\.*}.mp3" &
#wait for the ffmpeg encoding to finish before moving on
wait $!;
fi
if [[ ${lines[$i]} == *ogg ]]
then
echo "transcoding and moving file "${lines[$i]}
echo "filename is "$filename
ffmpeg -i "${lines[$i]}" -acodec libmp3lame -ab 192k "${countstr}_${filename%\.*}.mp3" &
wait $!;
fi
#specify input file to be the one specified on the command line (must be m3u format)
done