Archive for category Uncategorized

Is it catching on…?

Mark isgetting an internal server error on  I think I amgetting an internal server error too.

Unless, of course, I don’t get it.

(The first result got fixed, this version (courtesy Google’s cache) has the goodness.)

Music library status

In case anyone is curious, here’s a status update on my music library debacle.  I finally got to a point where I had a what should be a clean copy of my music repository on my NFS drive, and I just had to point iTunes at the cleaned repository…

I was resigned by this time to losing all my ratings and play counts, as it seemed that the only method that would work was to delete the existing library and recreate it from a hack of the XML version of the library file.  This was what I started when I got back from holiday.

I backed everything up, then edited the XML file and search-and-replaced the file paths in the entries with the new location.  Then, I deleted the “iTunes Library” file and started iTunes.  Sure enough, empty library.  I selected “Import…” and pointed iTunes at the XML file.  After a fair amount of grinding, it gave me an error saying that “not all the songs could be added as they could not be found”.  Sure enough, only about 1400 of nearly 2600 made it into the library.  My search-and-replace must have been incomplete… 🙁

I figured that rather than scratch the library yet again and try and find the 1200-odd entries in the XML file that I had failed to edit correctly, I’d just use “Add to Library…” to bring in the missing files.  This worked correctly, and my iTunes listing was back to its former glory (without play counts and ratings though).

Then the moment of truth — iPod time.  I had not plugged my iPod in at all after the library had blown up, so I was a bit nervous about the possibility of an iPod sync removing all the media and details from the iPod.  I was hoping though that iTunes would read the ratings and play counts off the iPod and restore them to the library.

I sat the iPod in its dock, and got an error about the iPod being synced to a different library.  Nuts.  I reopened the XML file and found a field called “Library Persistent ID” which was different between the original XML file and the new one — so unfortunately iTunes didn’t import the ID from the original file.

At this point I got a little blase about the whole thing.  I’d already trashed the library once, how much worse could it get?  I decided that this ID value had to be in the binary library file somewhere, so went looking for a binary editor that could help.  I found a very cool little program (with an equally cool name) called 0xED that allowed me to edit the iTunes library file — and lo and behold, right there in about the first 100 bytes of the file, was exactly the hex string that was in the XML file.  I shut down iTunes and substituted the old value into the binary library file, and restarted iTunes.  So far so good.  I plugged in the iPod…  and it synced!  The majority of the files were fine, there were a couple of podcast episodes and a few old miscellaneous files that had to be copied to the iPod.  Unfortunately though, the ratings and play counts didn’t come out of the iPod so are presumably gone for good.

So I’m working again, right?  Wrong.  This mess was the kick in the pants I needed to go away from using iTunes for music and switch to something Linux.  The main issue I have is with podcasts — I’ve never really been happy with the Juice/iTunes combination, since Juice splattered podcasts all over my library (it doesn’t put everything neatly into the “Podcasts” directory like iTunes did) — and I forgot the reason I switched from iTunes to Juice for podcatching in the first place.  I’ve got an idea in mind to combine a command-line cron-scheduled podcatcher with the MySQL backend for amaroK, but more on that when (if) I get something working.

iTunes library mismanagement

All I wanted to do was change the mount point that my music was exported on.  A simple change from one path to another on my NFS server.  What I’ve ended up with is a total train-wreck — my library is in a shambles.

I’m sure that in the past when I’ve relocated the library all I needed to do was select the new location in iTunes Preferences.  iTunes would recognise all the music in the new location and update the library.  This time, however, nothing I did would tell iTunes to look in the new location!  If I made the old location unavailable, it complained that all the music files were not locatable.  If the old location was present, iTunes just didn’t bother making any change to the index.  At one stage I told it to “Consolidate Library…”, one of the many recommendations I found on the Innerweb for handling iTunes Library moves, and it started copying duplicates of all my music files into the NFS share.

The worst part came when I told iTunes that it could manage my music library.  The Web and iTunes Help told me that with this option enabled, iTunes would keep my library organised according to artist and album detail.  Since my library was already quite structured I thought there would be little for it to do.  Unfortunately, iTunes didn’t think that my library was quite structured enough.  It started unwinding things like the “Compilations” directory — a directory it had created itself — into strict directories for the artists on a compilation.  I don’t have many compilation albums, but it was making an an absolute mess of a couple of albums that have songs by different artists.

So why was this a problem?  The problem happens because my library is shared between a number of applications — iTunes itself, ccxstream for sharing music to the XBMC-running XBoxes, and firefly (formerly mt-daapd) for streaming to DAAP-capable players (like Amarok and XBMC — I was looking at using DAAP as an alternative to running ccxstream).

So now I have a broken library.  All because I wanted to change /data/music to /lvs/music.  Why can’t iTunes manage that?  Why is it that iTunes feels it has to dereference the link to the music location, and if it thinks nothing has changed, not update anything?  As part of trying to get this working I had let iTunes move my library back off NFS to local disk, and I renamed the folder to try and hide it from iTunes as part of telling it that I had moved it.  When I went into iTunes preferences, the library location showed the name that I had renamed the folder to!

iTunes is obviously smarter than I am.  I should have not bothered trying to organise my storage the way I want to — once I let iTunes touch my network, I should have just let iTunes have its way.  So recommend all the iTunes fan-kids who posted some of the recommendations I read.

I’m much less keen to throw down on an iPod touch now, or even the Apple TV I was talking Susan into.  Maybe I’ll feel different after I’ve worked out how to put my library back together… or better still, worked out a way for me to manage music on my (and Susan’s) iPods without iTunes (my previous efforts there haven’t been successful).

An Open Letter to Australian Commercial Television Networks

From the television-viewing public of Australia: We applaud your attempt to bring us back from BitTorrent by airing popular US shows shortly after their original US airdates.  However, your gesture is hollow and insincere as long as you continue to treat us as having no intelligence to make informed decisions about our choice of entertainment.

We do not appreciate your insistence on devaluing your hard-won programming with the following:

* distracting and idiotic lower-thirds (believe us, in the middle of House we don’t need to be reminded when Neighbours is on.  If you intended to distract us from the programme, you succeeded — but we changed channel.);

* accelerated rolling or compression of credits to make room for promotion reels, destroying the readability of the credits (some respect for the folks that make the programs you use to make your money, please.  If you can’t take 30 seconds out to thank the folks that produce your programming, YOU’RE IN THE WRONG BUSINESS.);

* worse still, the replacement of the original production credits with your own high-speed microscopic version (see above.  Get a clue.);

* obliteration of theme music with continuous voice-overs that start from the first frame of credits until the last (in fact some of us can remember when a voice-over was exactly that, and you could still hear the original track);

* utter disregard for timeslots and scheduling, either by deliberate obfuscation (telling the guides that 8:30 is the start time, while the real start time is ten minutes later) or programming overruns (also likely deliberate.  If I didn’t choose to watch Big Brother, I’m hardly going to stay and watch it while you make the program after it late.);

* station logos that add nothing to the security or traceability of material (“This capture of Sea Patrol, Sir, from the logo we think it was captured by an AUSTRALIAN from a broadcast from Channel NINE!”).

Call it whatever you like: fast-tracked, streamed, straight-off-the-satellite, we don’t care.  Treating us like idiots is what’s driving us to torrents, not episode lag!  For many of us who value the experience of being entertained and not that of being marketed-at, we are very likely to WATCH THE TORRENT and then BUY THE BOX-SET DVD.

The sooner you reposition yourselves back to being useful and valuable components in the entertainment supply-chain, instead of annoying and costly middle-men, the better -off we all will be.

Some hints at how this might occur:

* Ditch the lower-thirds.  

* Respect your content producers, even those who are employed by your network.

* Setting your programming schedule is like making millions of appointments with your viewers.  KEEP YOUR APPOINTMENTS!

* Remove the logos.  I’ll admit that they’re less intrusive here than in some locations (or on cable) but they’re more harm than good.

Yours was an honourable industry that has been tarnished by overseas influence and corporate greed.  Only you have the power to reverse the trend and make Australian TV great again.

Website changes

Nothing visible at all (I hope) but I made some virtual site changes.  Some internal applications were actually visible externally, and I’ve made a new virtual host to isolate that stuff to the internal network (yes, I know I could have done it differently, but switching virtual hosts was easier then the first method I planned, which was to move the Internet-facing content to a different machine).  The main thing I wanted to achieve was a separation of logging — because my internal access to things like Nagios and Cacti was mixed in the log with external visitors, I could never get a clear picture of traffic ratios and so on (again, easily fixed by tweaking my log analysis software, but I like the virtual host method better).

Power over Ethernet for Fun and Profit?

I decided to go PoE to run a few of the phones.  So far only the 7970 is using it, as the switch I got is IEEE 802.3af PoE (because I didn’t feel like selling the motorbike so that I could afford a Cisco switch to run the older Cisco phones).  Not one to let mere electronics stand in the way of me running the Cisco pre-standard phones on 802.3af, I set about with crimp-tool and soldering iron to try and make it work.

It’s probably fair to say that success has been limited.

The problem arises because Cisco implemented a PoE mechanism prior to the IEEE 802.3af standard being ratified.  This method, not surprisingly, is completely incompatible with IEEE 802.3af.  It’s generally referred to as “Cisco pre-standard PoE” — Cisco gear that does PoE usually supports both their pre-standard and IEEE 802.3af.

(When I come across something like this I wonder if a lot of the cost of Cisco gear is soaked up by the added complexity of having to support both the “system they designed and rushed into the market to try and preempt the standards process” as well as the “system the industry agreed on and ratified because the right way is not always the Cisco way”.  But I digress.)

Thanks to some information like this, I found out how an 802.3af-compliant switch senses that it should supply power to the wire.  It’s quite simple: the switch looks for a “signature resistor” across the cable (follow the linky-trail for more info).  This meant it was quite easy to convince my switch that it should apply voltage.  (Susan was a little disturbed by how excited I got about making a little green light come on.)

Actually getting power into the phone is a different matter.  The page I link to above has about five different versions of fact and fiction when it comes to running the Cisco phones off 802.3af PoE gear, but it misses out on one critical piece of information (or at least it will until I get an ID there and update it).

The IEEE 802.3af standard defines the partners in a PoE transaction as the Power Source Equipment (PSE) and the Powered Device (PD).  The PSE can be either a 802.3af-compliant switch, or some kind of intervening device like a PoE midspan or injector.  The standard also defines two “modes” of PoE: Type A, where the power is supplied over the same cable pairs as the Ethernet signal; and Type B, where the power is supplied over the spare pairs in the cable.

The page says that with a resistor and a crossover-wired cable, you can  run a pre-standard Cisco device off an 802.3af PSE.  This is only partly true, because of a very important little piece of info that’s only alluded to on the page.  The critical info is this: while an 802.3af PD should be able to work on either Type A or Type B (as you don’t know what kind of PoE source you’re going to be connected to), a PSE can be Type A or Type B.  This is particularly important when it comes to the pre-standard Cisco phones, as they can only work as a Type B-style device — the Cisco pre-standard had no way to receive power over the data pairs.

This is why folks have success running Cisco phones off midspans and injectors — because they are 802.3af Type B devices.  Type B is used when you are injecting power along the cable run (i.e. without access to the Ethernet PHY).  They then crow on about how they got their Cisco phone working with 802.3af — as always, the devil is in the detail.  In the case of trying to run Cisco phones off 802.3af Type A devices like switches, you are left with the problem of extracting the 48V out of the data pairs without breaking the Ethernet link to the end device.  Not simple.

The page above lists one switch that appears to work with the crossover-cable trick: the Netgear FS726TP.  Knowing what I know about 802.3af now, it would seem that Netgear decided to make their switch a Type B PSE instead of a Type A.  Is it wrong?  Well no, but some folk may be surprised why they don’t get power over some cables when a different switch works fine.

The good thing about the Cisco pre-standard (if there can be a good side to it) is that it should be quite easy to rig up a DIY injector using the original power supply.  Since the phone expects power over the spare pairs, there’s no need to use an adaptor to split the cable out again at the phone end.  A DIY midspan using a single 48V PSU would reduce the losses in running a number of separate power bricks too.

So if there are budding Cisco PoE hackers out there, be aware of the need to know a little bit more about your 802.3af switch than what the manufacturer says on the glossy brochure. 😉

PS: While researching this, I came across a forum comment from someone who said that one of their pet hates was people referring to “Power over Ethernet” when it should be “Power over Cat5”.  Well, one of my pet hates is over-generalisation.  There can be no argument that the way that power is delivered in PoE is specific to the Ethernet wiring of Cat5-style cable.  Power over Token-Ring, if such a thing existed, could not be the same as PoE because different pairs of wires are used.  Likewise Power over ISDN U-Bus, Power over POTS, whatever.  My advice to Mr “Power over Cat5”: keep your generalisations to yourself, if you please!

iPod touch: Balance please

I re-read my post about the iPod touch and realised I probably wasn’t very balanced in the way I discussed it, particularly in light of the fact that I specifically said it wasn’t going to be a ra-ra post.  Maybe I’ve had a cooling-off period.  🙂  So, here goes with some of the negatives I can see…

It doesn’t have a radio, and doesn’t (as far as I know) have sound recording capability — these are a couple of features that many folks find important in a portable audio device.  Also, just because I was out of touch with my original estimate of the price, doesn’t mean that it isn’t overpriced.

16GB of storage, while impressive in some ways, is miniscule for what could be considered, thanks to the size of its screen, Apple’s flagship portable video display device.  A decent amount of storage, such as those now offered on the iPod classic, is going to be needed for a lot of people to take this seriously in comparison to something like a Creative Zen Vision or Archos unit.

Some might say the biggest criticism is the fact that, like the iPhone, there’s limited potential for third-party expansion.  Apple is a visionary company, but they can’t think of everything in advance and to not allow (or make it hard for) third-party applications to be delivered on these devices shows a distinct lack-of-vision.

Maybe I have cooled off on it, but I’m a little less keen on shelling out an AU$549 lump of money now.  Maybe Apple’s early announcement was a bad thing — they might have got a lot of impulse buyers just drop the money on it and then see the negatives, instead of (like me) having some time to think about it before being able to spend the dough.  Not that I think the iPod touch will be a failure, but given the US$200 price drop on the iPhone within six months of release I think I’ll hang onto the trusty-old third gen iPod a bit longer.

Authentication trouble

Here at the Crossed Wires Campus I’ve had LDAP at the centre of most of what the network does for quite some time now.  User-id management, telephone directory (integrated into the phone system), automount maps, Samba domain database; I even had DHCP running with LDAP as a backend for a while.  Most boxes in the house touch LDAP in some way every time they boot.  To demonstrate the multi-platform portability of that kind of configuration, I even had the Macs in the house able to log on user-ids that existed only in LDAP.  Until recently.

I don’t know the details of it, because it was something I only did occasionally to show that it still worked.  Now it’s stopped working, presumably after a Mac OS X update or other.  When I try and log on with an LDAP user, I get the wobbling password box.  That’s it.  The system logs tell little on the Mac, but on the LDAP server I get an error message about a failed SASL bind.

I’ve only ever set up SASL enough to support IMAPd, and even then it’s just talking to LDAP to do the work.  I use LDAP to store passwords, and for my purposes that’s always worked.  It used to work on the Mac too, but I can’t get him to stop trying to do a SASL bind to LDAP.

At about the same time as this, I was playing with a Jabber bot.  I read the instructions, configured appropriately, and it completely failed to function — its logon to the Jabber server was rejected.  Wireshark to the rescue — it was trying to use SASL to log on the the Jabber server.  Sure enough, my Jabber server was advertising SASL authmechs.  I removed the SASL settings (well, just the available authmechs) and the bot was able to log on.

So I started thinking if these issues were the kick I needed to set up a proper SASL and Kerberos system.  My ideal would be to get saslauthd to provide authentication service without having to go all the way to GSSAPI/Kerberos, something that should be possible…  except we’re talking about security systems here, so it seems that “The Right Way” is the only way.

The OpenLDAP documentation doesn’t discuss the SASL mechs PLAIN and LOGIN, since in their opinion they’re no different from LDAP simple bind.  Be that as it may, it would be nice to know how to do it!

DIGEST-MD5 is next, but the way it works you have to store user passwords in clear-text in LDAP (yes, clear-text passwords!) or use SASLDB2 to store passwords.  The former is unattractive, since I’m not so confident in getting an LDAP ACL right that would protect the password field from undesirable reading while still allowing it to be used, and the latter means I’d have to move everything to SASL auth unless I want to have password synchronisation problems (the very thing that moving everything into LDAP was meant to avoid).

Next comes Kerberos…  If I’m doing a heap of work to cut things to the DIGEST-MD5 SASL mech, might as well go all the way to GSSAPI, right?  That means more work, and again possible password sync issues between the Kerberos DB and those things still getting their passy from LDAP (although it looks as though using SASL you can tell OpenLDAP to consult Kerberos for password validation, so things using LDAP for password checking would actually get handled by Kerberos anyway).

One thing I thought to try was to rebuild OpenLDAP without SASL support — I’ve got a nasty feeling that since the last time the LDAP login worked on the Mac, I added “sasl” to the USE flags on the server.  Being built with SASL support means that slapd is offering it, even if it’s not set up (an ldapsearch for supportedSASLmechs verified this), and the Mac is seeing SASL auth advertised by the LDAP server and demanding to use it…  While a good theory, it’s not the problem.  The only difference in the log now is that there’s no message complaining about a failed SASL connection.

So after all that waffle about SASL, it looks like there’s something else happening. Likely something to do with the strange posixGroup entries it’s looking for called “ffffeeee-dddd-cccc-bbbb-aaaa-0000003c” and so on…

Back to the drawing board.

Cisco phones: To Brick And Back Again

Over the last week or so I’ve had some fun with a couple of Cisco IP phones.  I can say fun because it actually was this time: not only did I manage to recover a 7960 from a bad firmware flash, but I unbricked my 7970!

Via I found that a later release of the Cisco SIP firmware is in the wild, apparently with Cisco’s blessing.  I have a phone that I had originally set up in the bedroom, but it hasn’t been plugged in for a while, so I decided to drop the new firmware on it.

Pulled the files, put them on the server, changed the config files to point phone to the new version, then powered up the phone.  The loader upgraded, but then would not get an IP address from DHCP.  Argh.  Because it was in the loader (and not in full firmware) I had no way to manually set IP.  I traced the network, and it looked like the phone was simply ignoring the DHCPOFFER.

I straight away put the TFTP config back to the previous firmware version — as it seemed like the new firmware was bad, I didn’t want to risk the other phones in the house pulling the new version.

Of course I thought that there might have been a network problem, so I took advantage of the port-mirroring capability in my Linksys switch…  As far as it could tell, the DHCP response was definitely making it out to the phone.

I spent way too much time looking at DHCP options, the config of the DHCP server, and my switch setup (I tried VLAN tweaks and duplex settings in the switch).  A phone reset (holding down # while you power the phone up) didn’t help either.

I eventually found a reference to a way to do a full factory reset of the phone — after holding down # at power-up, press “123456789*0#”.  Knowing that this is what bricked my 7970, I was quite nervous about doing it.  But, since the thing ws effectively bricked already, I gave it a run.  The phone asked if I wanted to keep config, and I told it not to.  It then did a reset, the Universal Loader started, and successfully got a DHCP lease.  Because I had put the config on the TFTP server back to the previous firmware level, it reloaded the old level and before long I was back where I started.

Being a glutton for punishment, but more importantly because I knew how I could get out of the problem if it happened again, I switched the files on the TFTP server back to the new version and tried again.  This time it went off perfectly, and the new code went up without a hassle.  I went around and reset the other 7960s in the house, and they too went fine.

The only reason I can come up with for why the problem happened is that my DHCP server has changed since the last time that phone was on the network.  It’s possible that the loader kept some record of the previous lease; even though it was getting a proper DHCPOFFER from DHCP, the fact that it was a different IP or a different DHCP server meant that it threw away the offer.  Maybe.

Spurred on by my success in rescuing that 7960 from brickness, I decided to have yet another go at making the 7970 work.  Some time ago (before there was SIP firmware for the 7970) I bought it and never really got it working with Asterisk’s SCCP support…  Then when the SIP firmware came out I managed to obtain it, but in the process of trying to get it loaded I did a factory reset, which on the 7970 actually deletes the phone’s existing firmware load.  The SIP firmware would not load, because of a TFTP error during the download.  I’d removed the only functional firmware from the thing with the factory reset, so it was totally dead.  An ex-phone.

In the earlier days of the 7960, there was an issue that some older firmwares could not be upgraded directly to newer firmwares.  This issue affects the 7970 as well, and seemed to be the problem I was having with mine in getting the SIP firmware on.  The solution is to progressively load firmware versions to step the phone up to the latest version, but with no valid way to obtain firmwares from Cisco (and a philosophical objection to paying for the privilege, even if they’d sell it to me) I was stuck with a brick.

If only there was a web site around the Internet that had the files available for download…  And if only there was a search engine that could find those files on that server for me…  🙂  I found a firmware that the phone would accept, and which let me go up to the SIP firmware.  After some mucking around with the XML config file, I finally — more than two years after I bought it — have my 7970 online to Asterisk!

(one of the line names is obscured, just because)

I’ve even got some of the XML app stuff working on the 7970 (as it’s subtly different from the 7960, of course) but I’ll put that in another post.  I might even post some more screenshots!

Goodbye, Classy Miss

I left for a short trip to Adelaide last night.  I phoned home from the airport, and Nicholas didn’t want to talk to me… but Chelsea did.  🙂

Susan phoned this morning to report that Chelsea had been ill all night.  A hurried trip to the vet…  Examinations and X-rays revealed that Chelsea had bone fragments lodged in her bowel.  Minor surgery.  Apparently while she was under, more fragments were found.  Minor surgery suddenly becomes major.

She made it through the surgery, but died later in the afternoon.

How can the light that burned so brightly / Suddenly burn so pale?

We will miss you so much, Chelsea.