The Database

I needed a database that would allow the the Music Server to access information about Artists, Albums and Songs/Tracks in a simple and easy to access format. As already mentioned in the previous post the Web UI is written in javascript and working with JSON (http://json.org/) is probably the most forward data format available. If the database itself uses JSON for the data format life is made much simpler as there is really no marshaling to other formats required.

The core of the database is the songs/tracks file. This file contains around 8000 tracks in my music collection. To load this whole file into memory is really not feasible. The solution I came up was to use a basic form of indexing and access the file via the java.io.RandomAccessFile class. A track index is simply a length (length of the JSON track record) and an offset pointing to where it resides in the file. To access a track quickly all that is required is a simple data structure consisting of these two values.

The song/track JSON structure itself mimics the IDv3 MP3 tags that are read from the MP3 files. Here is an example of one :

{
  "album": "Legion of Boom",
  "author": "The Crystal Method",
  "date": "2004",
  "duration": 268,
  "mp3.bitrate.nominal.bps": 128000,
  "mp3.channels": 2,
  "mp3.copyright": false,
  "mp3.crc": false,
  "mp3.frequency.hz": 44100,
  "mp3.id3tag.genre": "Electronica",
  "mp3.id3tag.track": 4,
  "mp3.mode": "Joint Stereo",
  "mp3.original": false,
  "mp3.vbr": false,
  "mp3.version.layer": "Layer 3",
  "mp3.version.mpeg": "MPEG1",
  "path": "/Volumes/mp3/music/The Crystal Method/Legion of Boom/04 - The Crystal Method - The American Way - Legion of Boom.mp3",
  "title": "The American Way",
  "type": "mp3"
}

In addition to to the songs/tracks file there are a number of index files that associate track indexes to Artists and Albums.

There is an "Artists to Songs" index file. Each Artist is associated with an array of song indexes.

  "alice in chains": [
    {
      "length": 542,
      "offset": 1500107
    },
    {
      "length": 544,
      "offset": 1500649
    },
.....

There is an "Albums to Songs" index file. Each Album is associated with an array of song indexes.

  "...And Justice For All (mp3)": [
    {
      "length": 600,
      "offset": 3101248
    },
    {
      "length": 568,
      "offset": 3101848
    },

Also there is an "Artists to Albums" index file that handles finding the artist/album associations quickly.

  "arctic monkeys": [
    "Favourite Worst Nightmare (mp3)",
    "Whatever People Say I Am, That\'s What I\'m Not (mp3)",
    "Humbug (mp3)"
  ]

These 4 files made up the database for the initial version, however as development progressed I found that I needed to add 2 additional files :

• An "ArtistId To Artists" file
• A "Directory Information" file

The "ArtistId To Artists" file was required because of the discrepancies I found with the Artist names in my music collection. The would be a variety of combination that would ultimately match the same artist. For example "The Cure", "Cure" and "Cure, The". This id file ties the combinations together.

The "Directory Information" file came about when I wanted to add new music files to an existing database without starting from scratch. This file provides a snapshot of the directories providing the music files that can be used to compare what is being scanned off disk.

Writing the Music Server software

Having decided that I was going to write my own Music Server software from scratch I had to then figure out a number of core issues that were fundamental to succeeding.

1. Create some form of database of track information to allow easy selection via a user interface.
2. Decode and play the media files on the server and access track metadata (MP3 tags etc) to populate the track database.
3. Create a User Interface that was accessible via the devices I had available (PC based browser running on lo resolution NetBook and an iTouch)
4. Provide some form of service middleware to allow the UI to obtain data and control whats going on on the Server.

1) The direction I took was decided by what I planned to use for 3). The User Interface was going to be HTML based supporting and number of different devices. I'm pretty familiar with writing DHTML and Ajax based applications. With that in mind one of the easiest data formats to deal with in javascript is JSON. I decided to create a database around JSON structures.

2) As I was going to be writing java code the most straight forward answer was to use an implementation of the javax.sound API.  As luck would have it there is a very nice implementation available http://www.javazoom.net/index.shtml. Using JLayer, MP3SPI, VorbisSPI and Tritonus I was able to write java code that could obtain the metadata stored in the media files and also decode and play these files on the PC.

3) I'm very familar with the dojotoolkit framework (http://www.dojotoolkit.org/) and I have also contributed a javascript serverside templating framework (http://www.zazl.org/) that allows easy dynamic creation of HTML via DTL (Django Template Language). Using both of these together I was able to put together a flexible HTML Web UI that supports regualar browsers and also Apple devices such as the iPhone and iTouch.

4) As I was already writing java code in the form of a Web Application the solution for 4) was to write the service layer with servlets supporting a REST like API. The response format is JSON.

So what about a remote control ?

I now had the ability to play my music collection through my stereo as MP3 files but the PC connected to the amp was in my office while my speakers were in my family room where I wanted to sit and listen.

I needed a remote control that would connect to the PC and let me :

1. Select the tracks I wanted to listen to from the whole music collection.
2. Add the tracks to a playlist
3. Start, stop and pause the track currently playing
4. Select the next/previous track in the playlist.
5. Adjust the volume.

My PC is running Window XP. Probably the 3 most popular options for software to play the music were :

1. Windows Media Player (easily available from Microsoft).
2. iTunes (At the time I owned a Apple shuffle)
3. WinAmp (Free download)

To be honest none of the 3 offered any sort of remote control that I wanted to or could use, either because it wasn't free (WMP), required another device (iTunes using an iPhone/iTouch), or lacked the functionality to meet my requirements (WinAmp). I could use Remote Desktop or VNC to connect remotely from another computer but I could do better than that.

So I decided that it would be fun to write my own remote control. As I'm a software developer by profession writing my own was certainly not out of the question.

Now I could have considered writing a plugin for all of the options above but I felt that starting from scratch would be more interesting. I also wanted it to be something that could be run on different operating systems (as long as the USB DAC was usable via that operating system). I had been writing web based applications in Java for close to 12 years so writing it as a JEE Web Application seems to make the most sense.

Version 2 - Using a USB DAC

Having found that the quality of sound pumped through the amplifier via the PC's built-in sound card was far from ideal I decided to investigate alternatives. I could look into spending some money on a better sound board, preferably an external one where the noise interference should be reduced or I could look into buying a USB DAC. I didn't really want to spend much money and while researching options I found a pretty cheap one in the Silverstone EB01. The reviews for it were generally very good. Around the spring of 2008 I bought one for $85.

So I hooked it up to my PC and amplifier and found that the device was listed as USB speakers in the audio setup for the PC. The difference in sound between the build-in sound board and the USB DAC was night and day. The one annoyance was that the volume on the PC had to be set to the maximum and the volume controlled on the amplifier itself. This was (for me at least) an acceptable level of sound quality and I could always look into buying a better quality one in the future.

The next problem to tackle was now how to control playing the music on the PC in the office while I was in the family room.

Version 1 - The cheap route

Version 1 of my DIY digital music server was the cheapest by far ($0) but unfortunately was lacking in the sound quality department.

I had my amplifier (A&R A60 circa 1989) and desktop PC sitting in my office while my speakers (Linn Index II circa 1990) were in the family room. All I had to do was connect the PC to the amplifier and use the PC as a source. The PC was certainly no high-end machine but it did have built in sound provided by the mother board. There was a line-out jack that I could connect to a line-in on the amplifier. I already had a 1/8 jack to RCA cable (actually this was a little more complicated as the A&R A60 used DIN connectors, I had to also use a RCA to DIN adapter too) so I hooked the two together and tried playing a 128 bitrate MP3 via WMP. The sound was Ok but it certainly left quite alot to be desired.

After researching other people's experience with doing something similar it appeared that using a built-in sound card (or even a separate sound card) was not going to give the sort of results I wanted. I was going to have to look at other alternatives to hooking up the PC to the amplifier.

Introduction

Ever since I bought my first MP3 player and started to rip MP3's from my CD collection I have been interested in also making life easier for accessing my music on my home stereo. Many years ago I spent probably a whole week going through the process of inserting each CD into my desktop PC and ripping the contents into MP3's. This was back in the time when digital storage was expensive and  ended up ripping to a 128 bitrate. So now I had my full CD collection stored in MP3 format (I have since gone through the process again ripping to "flac" format as storage is so much cheaper) and transferable to my MP3 player via  Windows Media Player, iTunes or whatever software the MP3 player required. I could also play music via WMP through the powered speakers connected to the PC (a nice old pair of AR Powered Partners).

But what if I wanted to play music on my regular home stereo? My home setup is little unusual. In my house the family room is at the back while my office sits in the front. The speakers sit against a wall that is shared by the office. The amplifier and CD player sit in the office with the speakers connected vi speaker wire feed through two holes in the wall. The downside to all this is that controlling the stereo is quite problematic when sitting in the family room. Getting up and going into the office to change the volume, change CD etc gets old very quickly.

Also in the office is my desktop PC that has access to all the MP3 files. It seemed like a slam dunk to somehow hook up the PC to my amplifier to play the MP3 files on my home stereo.

This blog serves to detail the long and windy road I have taken to build a digital music server that is flexible and easy to use.