Car Hacking

With our trip to the Baltics just around the corner, I was contemplating the many kilometres of driving ahead and this soon had me recalling our trip across central Europe in 2006. In particular, I clearly remember how our sat-nav system thought we’d fallen off the edge of the earth when we entered Hungary. Map coverage in Czechia and Slovakia was also decidedly spotty. The latter country had no coverage at all outside of the capital, Bratislava.

In the case of the Baltic states, I knew that these countries weren’t covered at all by our system. That’s hardly surprising, because the DVD containing the maps identifies itself as being for western Europe.

That rather implies that there are other DVDs available for other regions of the continent, so I went looking on-line to see if I could find a reference to one for the area in which we’re interested.

I soon made the pleasant discovery that the 2009 Audi navigation DVD has an expanded country list that includes the Baltic states. The bad news is that Audi charges a ridiculous amount of money for an a copy of the uodated DVD: between €200-300, depending on how generous they’re feeling.

I paid this fee a couple of years ago, when Audi were offering a special deal, whereby they would update your MMI (Multi Media Interface, the car’s software user interface) to the latest version and supply you with the latest sat-nav DVD for a single, reduced price . This combination deal was being offered because Audi had just released a localised MMI translation for the Dutch market, along with spoken navigation directions in the Dutch language.

This isn’t the kind of money I feel is reasonable to demand for annual incremental improvements to the map coverage, however, so I decided that I would scout around on-line and try to find a copy of the 2009 DVD by other means.

As you might expect. it didn’t take very long.

Whilst looking for the item in question, I also discovered that my MMI was now long out-of-date; hardly surprising, really, since it had been two years since Audi performed the last one.

So, in addition to the DVD image I’d found, I also downloaded three CD images containing the latest MMI software. With information gleaned from a couple of Audi Web forums, I now possessed the necessary knowledge to initiate the update process.

Flashing your digital camera or PlayStation is one thing, but the prospect of bricking one’s car had me slightly more nervous than I would normally be at the prospect of installing new firmware on a piece of hardware. My nervousness only increased as the update got under way, with the software issuing grave warnings against causing any fluctuations in the supply voltage, for example by operating the windows or sun-roof.

Everything went smoothly, however, and I can report that our car is now running the very latest version of the MMI: 55.7.0 0835.

Immeasurably more pleasing, however, is that our sat-nav now boasts coverage of Lithuania, Latvia and Estonia. Quite how good that coverage actually is, though, will become apparent in a matter of a week or so. I suspect that only certain cities will be covered, with few, if any, interconnecting roads. Still, anything’s better than nothing.

While searching for information on Audi MMI software updates, I came across a couple of interesting postings that indicated that one’s battery charge level indicator could disappear from the MMI after applying an update. This was interesting to me, because mine had disappeared two years earlier when Audi applied the last update.

I had noticed this immediately and queried Audi as to the reason. They told me that the battery meter had been removed from the latest version of the software, because it was considered unreliable.

Imagine my surprise when I read in this Audi enthusiasts’ forum that the loss of the battery meter was actually an expected, if undesirable side-effect of applying the MMI update, and that an Audi technical service bulletin (TSB), issued at the time to guide official Audi mechanics through the specifics of the new software release, indicates that the battery meter is turned off by default in the new version. In other words, it needs to be reenabled after updating.

Not only that, but a second MMI feature that allows instrument cluster settings to be modified turned out to have disappeared at the same time as the battery meter. I never use that menu, though, so I’d never noticed that it was missing until I read about it in the forum.

The main point here, of course, is that Audi had carried out the update improperly. The two features in question should have been manually restored after updating the MMI two years ago. Does no-one RTFM these days?

Further reading taught me that the missing features can be restored without the intervention of an Audi dealer. One needs only a laptop, some software and the means to connect the laptop to the car.

Needless to say, the prospect of being able to hack my own car was one that I stoically managed to resist for all of 24 hours.

Within a few days, the required cable had arrived from the US and I excitedly headed out to the car to connect my laptop to the same port used by Audi dealers to connect their much more expensive VAS 5051/5052 diagnostic computers.

A few minutes later, the missing menu entries were restored. I also took the time to enable a hidden MMI menu that allows access to a number of low level MMI settings. I can get to this menu now by holding down the CAR + SETUP buttons for five seconds.

Finally, I ran a diagnostics check of all of the control modules installed in the car. Checking these is the first thing Audi does if you take your car to the garage with some kind of complaint. Of course, Audi charges by the hour, so time spent diagnosing problems costs you as much as time spent fixing them. Using my new gear, I can find out which modules, if any, are indicating faults, before I even call Audi to make an appointment.

As it turns out, I found multiple fault conditions that Audi had failed to clear on previous visits to the garage, including some caused by them when they had decoupled certain systems in order to work on the car.

Normally, one has no insight into this kind of thing and simply has to assume that everything is being performed expertly and by the book. The missing MMI features and the failure to clear fault conditions indicate to me that the situation at an average Audi garage isn’t any better than at any other place where one would hope to find technical expertise and methodical work practices.

As usual, if you want a job done well, you have to do it yourself. There’s a very real limit to my expertise with the car, of course and tt ends well before the electronics turn into the mechanics. It’s not as if I don’t need the garage any more. On the other hand, I have, at least, managed to fix the problem that I set out to fix.

If I had called Audi about the problem, it’s possible they would have continued to claim that the menus in question were no longer available in this release of the software. It’s equally plausible that they might have been reluctant to hook up the car to their computer to fix the problem. And, of course, it’s not unthinkable that they might have simply been unable to fix it, due to the same lack of knowledge that caused the problem in the first place. It would be an awkward conversation in which I had to refer Audi to their own internal bulletin in order to educate them about the solution for my problem.

From my little excursion into the inner workings of the car’s brain, it’s clear that a good mechanic has to be at least as competent in the field of system administration. That’s hardly surprising, of course, because for years now, computers have been taking over more and more functions of the car and an increasingly large number of faults can now be diagnosed with and fixed in software alone.

Unfortunately, the ever-expanding skill set required to maintain the modern car obviously isn’t something that we, the consumer, can take for granted. For that reason, being able to make simple changes in software and run diagnostic checks is a very welcome addition to the home toolbox, to say nothing of the satisfying hacker experience of hooking up a laptop to your car.

Hacking The ReadyNAS

As you may know, I use a Netgear ReadyNAS RND4410 for my internal network’s mass storage. I’m very happy with the device, but it is a bit inflexible when it comes to back-ups.

The problem is that most forms of back-up that it provides for don’t support the deletion of files that are no longer on the source. For example, if file A and B were backed up last night, but file A was deleted on the source by somebody this morning, tonight’s backup should back up only file B (and even then, only if it has since changed) and delete file A from the back-up destination.

This level of control calls for the common free software program, rsync, to be used for back-ups. The ReadyNAS does support rsync, but Netgear’s interface to it via the Web-based FrontView software is less than ideal. That’s because it doesn’t allow one to specify the use of rsync’s many optional flags and parameters. In particular, it doesn’t allow the essential --exclude flag to be used to omit certain directories from the back-up. I’m not being picky; I actually have a back-up job that will fail without the use of this flag.

Happily, though, the ReadyNAS is Linux-based and Netgear nowadays provide an EnableRootSSH patch, which will, oddly enough, allow you to ssh into your ReadyNAS as root.

I’d been resisting the temptation to do this for some time, because I had no good reason to do so. The ReadyNAS is sold as an appliance and one isn’t really supposed to go prodding at its internals. The potential for rendering one’s device non-functional is definitely there. Of course, I know what I’m doing (famous last words, I know), but I still have a healthy respect for devices supposed to operate as black-box appliances.

Nevertheless, I needed more flexible rsync functionality for my back-ups. An alternative to poking around on the Netgear would be to schedule the back-up jobs on the clients themselves, pushing the data to be backed up to the ReadyNAS instead of having the ReadyNAS run the back-up job and pull the data from the clients.

I wanted my back-ups centralised, however, so I installed the EnableRootSSH patch and went gently wandering across the file-system.

I found what I needed and was able to add the functionality I needed with 15 lines of Perl. Now, it’s possible to define a set of extra options to be passed to an rsync back-up job when it’s invoked.

I’ve posted details of how to do this to Netgear’s ReadyNAS forum, so I won’t repeat them here. I mention the hack here only to gain a bit of publicity for it, as I’m sure I’m not the only person who needs this extra functionality.

Of course, a much better solution would be for Netgear to integrate this into their FrontView Web-based interface. I’d much rather be able to use the supplied tools than have to resort to hacks like this.

Still, at one level, it is nice that Netgear have allowed this kind of thing to be done. It encourages experimentation, development and user community growth.

Please Rescue My Stagnant Software

In the 2.5 years since I left Google, stopped working and moved away from Silicon Valley, my motivation to write code has waned and reached an all-time low ebb.

There are many reasons for this. For example, I’m father to a busy toddler these days, I’m no longer surrounded by brilliant geeks who fire my imagination and make me research new technology, and I rarely happen upon a need for a piece of code that hasn’t yet been written.

Indeed, with very few computer systems to set up at home and no professional programming assignments to send me off on weird and wonderful coding tangents at home, the only piece of code that has seen regular development over the last year has been my TV guide data grabber for the Dutch cable TV network, UPC. Unless you were a Dutch MythTV user whose cable company was UPC, you wouldn’t have noticed this. It’s not exactly been my top download.

Even maintenance of my old projects has pretty much ground to a halt. You see, most open source software is born of a personal need. Necessity is the mother of invention, as they say. If code to solve a certain problem didn’t exist, I had to invent it, assuming it was within my capability; which it fortunately sometimes was.

Ruby/LDAP is a good case in point. Although I didn’t start Ruby/LDAP from scratch, it amply illustrates what I’m talking about.

After submitting a series of patches to Ruby/LDAP in 2003 and becoming frustrated at their slow rate of inclusion, the then maintainer tired of my e-mails and offered me the role of maintainer of the software. He had lost interest in the project, because he had long since stopped working with LDAP. See what I mean?

Anyway, I jumped at the chance. In the next year, development was rapid. I added a lot of features and fixed a legion of bugs. The development roadmap was easy. The use of LDAP at Google was exploding at the time, much of which was happening under my guidance and supervision. Every time a new need was found that logic dictated should be solved at the library level, it went on the Ruby/LDAP to-do list and was implemented by me not long after. I didn’t need to spend much time thinking about which new features should be added, because they simply presented themselves to me in the course of doing my work.

These days, things are a little different. Not only do I no longer have a need to manipulate vast quantities of data in LDAP directories, I no longer even have ready access to working LDAP servers full of real, live data. Granted, I could set up a server at home, populate it with junk, and use that to test my library against, but without real-world problems occurring to me, it’s not obvious which course development should take. In this scenario, the only real work that happens is the fixing of bugs.

It’s the same with most of the software I have produced in recent years. The conclusion is simple. Once I cease to use a particular technology, my interest in it eventually drops below the level required to continue maintaining software. Sometimes the technology is no longer used because I am no longer in a professional environment (Ruby/LDAP, Ruby/CorporateTime), but it can also be because I have ceased to use it at home (bash completion.

Another reason software development can grind to a halt is because it reaches the point that it satisfies all of my personal needs (Ruby/Password, Ruby/DICT, Ruby/Finance, acoc, the very ancient signature.

The upshot of all of this is that most of the software that is under my wing for maintenance has badly stagnated.

In spite of promises to the contributors and enthusiasts of my various projects, I have languished and failed to make available public source repositories, mailing lists and the like. Such resources would have allowed the development of the software to continue, in spite of my current apathy.

Part of the problem has been in not wanting to let my babies go. bash completion, in particular, was started by me from scratch and has been very successful. Whilst I haven’t made a cent from it, it features in many Linux distributions, has been in the Freshmeat most highly rated projects top 10 for many years and has featured in several magazine articles. Not bad for a piece of code that I started just to keep myself busy in the evenings, whilst killing time for a month in Canada, waiting to re-enter the US (it’s a long story).

Rightly or wrongly, I’m quite proud of that software, but that pride has got squarely in the way of its continued development. In not wanting to relinquish the reins of control and any vague geek recognition that might have afforded me, I have instead allowed it to stagnate. Its last release was over 18 months ago.

Well, I have seen the error of my ways and am now prepared to act. As of this moment, I am declaring all of my software eligible for adoption by new parents, with the exceptions of my MythTV data grabber and Ruby/Amazon. This latter project has, in spite of everything I’ve said above, seen some recent work and I still intend to release a new version within the foreseeable future (but please don’t hold your breath).

In particular, bash completion and Ruby/LDAP urgently need loving new homes. If you are reading this and would like to become the new maintainer of either project, please get in touch. Be forewarned, however, that you will need a fairly convincing argument to become the new owner of bash completion, as it needs a skilled and dedicated developer (yes, yes, even though I haven’t been that myself recently). Ideally, you will have been a reasonably prolific past contributor to the project, thereby having already proven your candidature.

If I don’t find new homes for these waifs and strays soon, I will scour my old e-mail for likely candidates and approach those myself. In fact, I may do this anyway.

So, it’s open season on my old software. Let me know if you’re interested, explain why and where you plan to take future development, and you could find yourself the owner of a mouldering CVS repository.

Recording Good Films

I found out at the weekend that we’re soon to be offered ten days of free viewing of the Film1 film and sport channels as a promotional stunt. Now, I have no interest in the sport channels, but there could be some good films on the film channels.

Unfortunately, during most of the ten day free period, we’ll be in the UAE and Oman. So, how to ensure that we don’t miss any good stuff?

Enter the latest version of tv_grab_nl_upc, 0.6.1, which is the grabber I wrote to feed my MythTV system with programme schedule data for UPC’s digital television network. This new version is able to look up the IMDB viewer rating for each of the films that it finds. This rating is actually a very good indicator of the true quality of the film in question, as it reflects the opinion of real viewers and, usually, with sufficient quantity that a reliable average results.

The only thing that remains to do is produce a custom recording rule in MythTV; and here it is:

program.stars >= 0.75 AND MONTH(program.starttime) = 2

AND DAYOFMONTH(program.starttime) >= 16 AND DAYOFMONTH(program.starttime) <= 25

AND channel.callsign LIKE ‘F1%

This says to record any programme that has a star rating equal to or higher than 0.75 (equivalent to a 7.5/10 rating on IMDB), when the start of the broadcast is in February, between the 16th and the 25th of the month, and the broadcast channel is any of the Film1 channels.

Using tv_grab_nl_upc and the above rule, we should have a few decent films to watch when we return from holiday.

Again, I find the above a great example of the power and flexibility of MythTV. No other PVR gives you this level of control.

Christmas specials

I noticed today that our MythTV box was going to fail to record the Christmas special of The Real Hustle. Why? Because the programme’s name was The Real Hustle Christmas and we had instructed the machine to record only The Real Hustle, which as far as MythTV is concerned, is an entirely different programme.

Now, one doesn’t want to have to create a second companion rule for each recording rule, purely to catch the Christmas specials. Nor does one want to have to create a power rule instead of a normal rule, so that one can match the basic programme name followed somewhere by the word Christmas.

What does one really want to do? Ideally, you want to record any programme that has Christmas in the title, if, when the string Christmas or Christmas Special is removed, the remainder of the title matches the name of any programme for which one has already created a recording rule.

To do this, create a custom rule, perhaps called Christmas Specials. In the search phrase, place the following:

program.title LIKE ‘%Christmas%’

AND REPLACE(REPLACE(program.title, ‘ Special’, ”), ‘ Christmas’, ”) =

REPLACE(record.title, ‘ (Power Search)’, ”)

Dissecting this, we first check to see whether a given programme in the TV guide contains the word Christmas in the title. If it does, we remove the suffix Special, if present, from the title. Next, we remove the suffix Christmas from the title. That leaves us with the stub of the programme’s title, which, with a bit of luck, will be the same as the title of a normal episode of the programme.

On the right-hand side of the equality check, we join with the record table to see whether there’s a recording rule with the same name as the title stub of the Christmas programme. The REPLACE function first removes the text that MythTV itself adds to the name of any rule that is a power search, as this will otherwise potentially cause valid matches to be missed.

If we find a match, bingo, the Christmas special is caught in the net and MythTV will record it.

As I’ve said before, absolutely anything is possible with MythTV’s recording scheduler, as long as you can figure out how to express yourself in SQL. This is the first time I’ve found a use for a database table other than program.