beatworm.co.uk

There is a top level navigation menu at the foot of the page

Scripting iTunes for the iPod, with Perl and Mac::Glue

I acquire new music at a relatively steady rate. I've been use my computer as a music library, storing my purchased music on a fileserver. Once I migrated to Macintosh, I started using iTunes to manage this process. It's a useful solution, what small trouble I do have is usually related to my unusual configuration. I keep my music collection on an NFS filesystem, enough of a weird thing to do that I'm surprised iTunes doesn't have more trouble with it.

A few years into this process I decided to get a portable audio player. An iPod was the obvious choice, despite the cost. Plug and go, lovely interface, Just Works™, all the usual. The 40GB model I went for was larger than my entire music library, compressed, to mp3 or increasingly to aac, with moderate settings. Synchonising is almost magically simple; set the iPod to sync 'checked items' and uncheck anything in iTunes you wish to exclude. I extolled the benefit of this approach to anyone who asked.

This approach will only scale so far. After several years of acquisition, the 'checked' set must exceed the 40GB capacity of the device. Luckily, by the time I reached it, Apple upgraded the iPod range, the new 'classic' guise, offering a potential 160GB storage. I toyed with one of these on display in a shop, but quickly gave it up unconvinced. I had misgivings about the new unit, foremost was the surprisingly sluggish interface.

In this latest generation, the menus have been changed to make more use out of the hi-res colour display; album art and wallpaper is displayed 'behind' the menu, on a vertically split screen, and it's clunky. The cursor and scrolling lag perceptibly behind the click wheel. The text data for the menu on my old iPod are small, and probably cached in RAM at start up. The new system would have to read the graphical data for the current selections dynamically. I'd guess that the hard disk design, saddled with a slower seek time and a pressing need to spin down to a power conserving state, struggles to deliver this as smoothly as the flash-RAM based units.

I put my thinking cap on. As my existing iPod approached capacity, navigating the menus to choose became tricky. Although responsive, just scrolling through thousands of entries on a small screen was a chore.

Then iPod touch range arrived. They seemed like a pointer to the future. The new iPhone interface software, with a respectable, solid state capacity, a responsive cover art browser, multi-touch, WiFi, and the host of iPod features my hard disk model predated; gapless playback, photos, video, TV out. With a SDK on the wayit was an development investment, it would excuse me from compulsively signing up for a UK iPhone when they shipped. Convincing myself thusly, I bought the 16GB touch the day after they launched; plenty of room for the music I actually needed to carry, if I could come up with a cunning way of managing it all.

Thinking about my iPod listening, in practice I was already using mostly a subset of my collection; a few dozen newest albums, along with another few dozen solid favourite albums. Whatever was the current obsession, was receiving the heaviest rotation, and more often than not this would be a new purchase.

'Album' is a key term here. I'm old, and this is mostly the way I listen to things. An archaic concept to the post-CD generation, but I grew up with the cassette-based personal stereo, and expect a complete album, in track order. I rarely skip the filler. I do not shuffle, aside from the occasional album shuffle when I'm too lazy to choose.

Manually managing the library was never an option. Although iTunes has an effective interface for large sets, spending time organising, and selecting albums by hand is not really complementing the 'pick up and go' nature of iPod, which is so much of the appeal. For data processing, though the ideal is always to make the computer do the tedious work. You can restrict an iPod to only sync from certain playlists. iTunes has 'smart playlists', a simple way to make dynamic lists of tracks. I have built some to help sort through my library. Smart playlists would be the easiest way to my dream of a smart library, if it weren't for the fact that they are so frustratingly track-based about their criteria. Trying to do anything sophisticated with in terms of complete albums quickly degrades.

There is a third way though. iTunes, like pretty much everything on a Macintosh, is comprehensively scriptable. If I could define a recipe to identify a good selection of albums, I could probably write a script that built a playlist I could use to synchronise.

The first step is a master playlist that filters music albums from my library, excluding podcasts, sound clips and other such flotsam. This is easily done, a smart playlist selects based on podcast, genre, and a few custom groupings and exclusions. Next I need a recipe to select a candidate list of albums from within this set, following the outlines I describe earlier. My n newest albums, all of my current favourites, and then a sample from everything else, so I have a few surprises to carry around. Ideally the sample will biased, weighted to favour things I like, but still rotating.

Newest additions are straightforward. As I import entire albums at a time, a smart playlist selecting from the master playlist limited by date added. iTunes 7 added an 'album rating' field, which scores entire albums, much like the original 1-5 star track rating for individual songs. The rules to select the dynamic album list can derive from this; all the albums with a five star rating. and then Yalbums rated four stars, and Z albums rated three stars, and so on, where Y > Z.

And on to the final hurdle. iTunes might be fully scriptable, but the default language for the job is AppleScript, which is something I really struggle with. It is an unusual programming language, more loosely structured and irregular than most, based around a "user friendly" pseudo-english syntax. This may be a good way to present programming concepts for the non-programmer, I really cannot tell. As an experienced programmer, it baffles me. I find it obfuscated and impenetrable. But all is not lost. It so happens AppleScript is implemented around something called OSA, which exposes an event driven model of messages applications via an API, which various scripting languages can be built around. Naturally, there's good perl support for this infrastructure. I can do perl.

Although the raw apple events based mechanism is fairly ugly, and unintuitive, there's a great module on the CPAN, called Mac::Glue, which binds the English like keywords from the application's scripting dictionary into a somewhat more intuitive API. The message syntax remains a close relation to the AppleScript, so you can use those tools to inspect the command dictionaries, and crib syntax from the internet and the help files, yet work with traditional perl idioms for control and data structures.

A slight drawback to Mac::Glue is a need to generate a 'glue file', using the supplied 'gluemac' tool, for each application that you want to script. These files require rebuilding if you re-install the application, a minor irritation for something like iTunes, which is periodically reinstalled by Software Update. A pure AppleScript would not suffer from this fragility, but then a pure AppleScript would have taken me umpteen times as long to prototype and debug.

Anyway, here is the script I came up with.

#!/usr/bin/perl -w
use strict;
use warnings;
use diagnostics;
use Data::Dumper;

use Mac::Glue ':all';
our(%albums);

my $itunes = Mac::Glue->new('iTunes');

# get the playlist with only music albums in it, and toggle it's shuffle
# to randomize the order we will read tracks in

my $list = $itunes->obj(playlist => whose(name => equals => 'music albums'));
$list->prop('shuffle')->set(to => 0);
$list->prop('shuffle')->set(to => 1);

my $playlist_id = $itunes->prop('index', $list)->get; #explicit form of $list->prop

# aref pairs are album rating, number of albums required
for my $spec ( [60,20], [80,30]){
  my ($rating,$required) = @$spec;
  my $tracks = $itunes->obj('track' => gAll, 
                            tracks => whose(album_rating => equals => $rating ), 
                            playlist => $playlist_id);
  my @albums = $tracks->prop('album')->get;
  my @artists = $tracks->prop('artist')->get;
  while ($required){ 
    # record every different album name, until required is decremented to 0
    my $title = shift @albums; 
    my $artist = shift @artists;
    next if exists $albums{$rating}->{$title};
    $required --; $albums{$rating}->{$title} = $artist;
    
  }
}

my $target_list = $itunes->obj(playlist => 
                               whose (name => equals => 'Album Shuffle'));
$playlist_id = $target_list->prop('index')->get;

# clear target playlist of all tracks
my $tracks = $itunes->obj('track' => gAll, playlist => $playlist_id);
for my $t ( $tracks->get ){
 $t->delete;
}

# we now have a data structure like this  
# ( album_rating => { name => artist, 
#                     name2 => artist2, 
#                     name3 => artist3...}, ...)



my $target = $target_list->get;
foreach my $score( keys %albums){
  foreach my $album( keys %{$albums{$score}} ){
    print "adding $album\n";
    my $tracks = $itunes->obj('track' => gAll, 
                              tracks => whose(AND => [album => equals => $album],
                                              [artist => equals => $albums{$score}->{$album}] ), 
                              playlist => 1 );
    my @files = $tracks->prop('location')->get;
    map { $itunes->add($_, to => $target )} @files;
  }
}

I use a normal, 'dumb' playlist called 'Album shuffle', that I use to collect the album selections. Because I have my shuffle settings set to 'Albums' in my iTunes playback preferences, toggling the shuffle parameter on the master albums playlist randomly sorts this by album. The script then walks over the master list from the top, collecting groups of tracks grouped into score buckets by album and artist, until I've collected the required number of albums in each bucket; currently it's hard-coded to 20 three star albums, and 30 four star albums. Next I delete all the tracks on the playlist, and then walk through my bucketed albums, grabbing all the tracks for each album in turn from the library, and then adding each track to the 'Album Shuffle' playlist.

It's quite a Heath Robinson approach, and not without drawbacks, but it does get the job done.

The main problem is efficiency. AppleScript does seem to carry a significant overhead, and driving it via perl is slower still. If I was going to optimise it I'd guess that porting it to native AppleScript would probably help a bit. Finding a way to bulk add tracks would almost certainly improve the expensive 'track at a time' loop, and rather than deleting then replacing everything periodically, it might make as much sense to refine the script to selectively remove and rotate a subset of the playlist more frequently.

As it stands, although the time to build and resync probably runs into an hour or so, and consumes significant CPU while it goes, in practice it's not a huge problem yet, because I only run it once a month or so. Thinking like that, it is possibly a candidate for a cron or launchd job, which would make it even less intrusive.

This entry was posted on Thursday, January 10, 2008 at 15:44 in computers, music, programming.
You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

3 Responses to “Scripting iTunes for the iPod, with Perl and Mac::Glue”

  1. beatworm.co.uk » Blog Archive » iTunes automation, revisited Says:

    [...] week , which introduced some excellent new features , such as Genius playlists, but broke my fancy perl script which I wrote to rotate my music library on my iPod [...]

  2. Jon Says:

    You can add files to a playlist using the duplicate method rather than adding them as a file. This seems to be a little quicker as iTunes does not re-read the meta data:

    my $track = $itunes->obj(track => 1, tracks => whose(persistent_id => equals => $track_id), playlist => 1);
    $itunes->duplicate($track, to => $playlist->get);

  3. cms Says:

    @Jon – Thanks, yes. I did discover the 'duplicate' method a little later on, and I've incorporated it into my new version of the tool, which I re-wrote in Python (mostly because I was investigating Python, and an iTunes update stopped my perl script from working). There's a link to that version of the tool in the first comment above. Thanks for explaining why it is faster, I do find applescript a bit of a cryptic affair!