Hacking The ReadyNAS

Posted on Tuesday, 30th September, 2008 by ianmacd

As you may know, I use a Netgear ReadyNAS RND4410 for my internal network’s mass storage. I’m very happy with the device, but it is a bit inflexible when it comes to back-ups.

The problem is that most forms of back-up that it provides for don’t support the deletion of files that are no longer on the source. For example, if file A and B were backed up last night, but file A was deleted on the source by somebody this morning, tonight’s backup should back up only file B (and even then, only if it has since changed) and delete file A from the back-up destination.

This level of control calls for the common free software program, rsync, to be used for back-ups. The ReadyNAS does support rsync, but Netgear’s interface to it via the Web-based FrontView software is less than ideal. That’s because it doesn’t allow one to specify the use of rsync’s many optional flags and parameters. In particular, it doesn’t allow the essential --exclude flag to be used to omit certain directories from the back-up. I’m not being picky; I actually have a back-up job that will fail without the use of this flag.

Happily, though, the ReadyNAS is Linux-based and Netgear nowadays provide an EnableRootSSH patch, which will, oddly enough, allow you to ssh into your ReadyNAS as root.

I’d been resisting the temptation to do this for some time, because I had no good reason to do so. The ReadyNAS is sold as an appliance and one isn’t really supposed to go prodding at its internals. The potential for rendering one’s device non-functional is definitely there. Of course, I know what I’m doing (famous last words, I know), but I still have a healthy respect for devices supposed to operate as black-box appliances.

Nevertheless, I needed more flexible rsync functionality for my back-ups. An alternative to poking around on the Netgear would be to schedule the back-up jobs on the clients themselves, pushing the data to be backed up to the ReadyNAS instead of having the ReadyNAS run the back-up job and pull the data from the clients.

I wanted my back-ups centralised, however, so I installed the EnableRootSSH patch and went gently wandering across the file-system.

I found what I needed and was able to add the functionality I needed with 15 lines of Perl. Now, it’s possible to define a set of extra options to be passed to an rsync back-up job when it’s invoked.

I’ve posted details of how to do this to Netgear’s ReadyNAS forum, so I won’t repeat them here. I mention the hack here only to gain a bit of publicity for it, as I’m sure I’m not the only person who needs this extra functionality.

Of course, a much better solution would be for Netgear to integrate this into their FrontView Web-based interface. I’d much rather be able to use the supplied tools than have to resort to hacks like this.

Still, at one level, it is nice that Netgear have allowed this kind of thing to be done. It encourages experimentation, development and user community growth.

Posted in Hacking, System Administration | 5 Comments

Helpdesk Staff Wanted For Shit Work

Posted on Tuesday, 30th September, 2008 by ianmacd

That’s the sign I saw in a temp agency’s window today, here in Brussels.

At least, that’s what I thought it said. The actual text was: Helpdesk staff wanted for shift work.

I can’t imagine how I made that mistake.

Posted in Travel | Leave a comment

Return To Form

Posted on Tuesday, 30th September, 2008 by ianmacd

Killing Joke last ruled the roost last night.

The Ancienne Belgique is a good venue, a fact that becomes evident before you even reach the main hall. You can dump your belongings in a locker downstairs, so there’s no need to queue for a cloakroom ticket.

The various tour T-shirts were mostly sold out in my sizes, so that made the decision whether to buy one much easier. I have a wardrobe full of T-shirts, but it’s an old habit of mine to buy a shirt from each gig that I go to, just so long as the shirt lists the tour dates.

In the main hall, the support band were playing. It was some metal combo, but they were rather good. At the very least, they served to indicate that the venue had good sound.

The Joke took to the stage at 21:00, looking ready and in the mood. Without any ado, they launched into Requiem. The sound was crisp, clear and very, very loud. Around me, younger members of the audience, who weren’t even alive when this material was first released, hastily shoved yellow ear-plugs into their head. The youth of today…

The onslaught of the eponymous first album continued all the way through to Primitive. There were scarcely any gaps between the songs for the band to draw breath. The sound really thundered out of the PA, turning an unremarkable studio track like Bloodsport into an earth-trembling, chainsaw guitar wall of sound. You didn’t have to dance: the earth moved under your feet.

With no pause for thought, What’s THIS For…! continued the aural bombardment. Somewhere in the middle of it, Jaz announced “a change of plan”, at which point the band unexpectedly treated us to a stomping version of Eighties. I couldn’t tell you why, because it’s a track from several years later.

It was interesting to hear tracks that I’ve never heard live before, such as Madness and Who Told You How? And not just that, but also the fact that they were being played by exactly the same line-up that had originally recoded them.

During the encore, Jaz said a few words about dearly departed Paul Raven. He said that Paul would “always leave the band when the money ran out; like always!” and added, “But he would always come back if you waved more money at him.” Finally, he concluded “the world is a darker place for his loss”.

This was the prelude to another deviation from the period being aired tonight, as the band launched into what is perhaps their best known song, Love Like Blood.

Change was a particular highlight of the evening for me. I haven’t heard it sound so good since seeing the band in San Francisco a few years ago.

Needless to say, I’m looking forward to tonight’s gig, which will see the band fast-forward about thirteen years to the Pandemonium period. Apparently, that’s the band’s favourite album, but it’s also the album that Youth returned to the fold to record. Besides that, it’s one of my favourites, so it’s going to be a real treat to hear the entire album played live.

Posted in Music | 2 Comments

To Brussels For A Joke

Posted on Monday, 29th September, 2008 by ianmacd

It’s been silent on the blogging front of late, as you may have noticed.

You may thing that means not much has been happening, but in fact, the opposite has been true. Something had to give, so there wasn’t any time to blog.

I have plenty to write about, but still no time to do the actual writing.

This entry is also being squeezed in before I head off to Brussels in the car for a two night stint of Killing Joke at the Ancienne Belgique.

The band aren’t coming to Amsterdam this time and, well, this time it’s a bit special. With the recent sad death of bassist, Paul Raven, the band have reunited with the original line-up that hasn’t played together since 1982. That means Youth on bass and Paul Ferguson on drums. Apparently, the idea of the reunion was hatched at Raven’s funeral.

Not only that, but they’ll be playing their first two albums tonight in their entirety. Tomorrow, they’ll play the full Pandemonium album, plus the singles recorded for Island in 1979 to 1980. So, it’s not your usual band standard mix of songs through the ages, plus the requisite crowd-pleasers. No, this time, we get the seminal early material through to 1981, plus the band’s favourite album, Pandemonium.

I’d rather not have to spend a night away from home — I miss my children if I’m away from them for more than about four hours — but, like I said, this is a bit special. I also had to ask myself, ‘Which bands would I still make the effort for? If not for the Joke, then for whom?’

I once missed a red-eye flight to Texas, where I would have attended the 2003 Ruby conference. Instead, I stayed on at Slim’s in San Francisco to see the end of Killing Joke’s performance there. That made it a pretty expensive ticket, but in return, it was a hell of a gig.

Besides, there are few people as angry as Jaz Coleman when he’s on stage. Aged 48 now, he still really means it. As I hurtle towards the grave myself, I don’t notice any diminishing of my own levels of bile and gall. In Jaz, I find a kindred spirit and I like the hellish noise his band makes, too; once described by Paul Ferguson as “the sound of the earth vomiting”.

And so I head to the Belgian capital this afternoon for a night of cathartic spleen venting. And then a good night’s sleep, a walk through the old city, a few photos, some good food, coffee, and a second night of purging oneself in the company of like-minded people.

At least, I hope they’ll be like-minded people. They may just be a bunch of dispassionate, complacent zombies, the like of which I’ve suffered before at concerts on the European continent.

But with the US about to elect another misguided fool to continue the flow of billions in military aid to Israel and prolong America’s self-assumed role of policeman to the world, imposing its own peculiar brand of pseudo-democracy in the pursuit of self-enrichment, there’s still a surfeit of things to be angry about in the world.

So, perhaps now, more than ever before, the world needs Killing Joke. I know I do.

Posted in Music | Leave a comment

Don’t Interrupt Me

Posted on Wednesday, 27th August, 2008 by ianmacd

UPS: no, not the company that delivers parcels all over the world. I’m talking about Uninterruptible Power Supplies.

It was about time the house had one, so I took the plunge and purchased one. In fact, I did this about two weeks ago. It arrived a few days later and was still lying in its box in the hallway until yesterday.

It wasn’t just laziness that kept it in its packaging. It’s an APC Smart-UPS RT5000 XL, which is quite a heavy-duty unit. As such, it needed its own heavy-duty electrical group in the house, which meant that a qualified electrician had to come to perform the work.

One of the parts that the electrician needed for the job took a week to arrive, so it wasn’t until yesterday that the work to install the UPS was completed. At last, the UPS, a hulking great thing weighing in at around 60 kg, could be lugged downstairs to the cellar and installed in the cupboard under the stairs. More bright LEDs to peer back at me from the darkness!

I seized the opportunity to have the electrician rewire one of the other electrical groups, so that one of the sockets in my office is now routed via the UPS. This means that my desktop computer, monitor, telephone and fax can all connect to it, which saves me from having to buy a separate unit for the office. That’s very cool.

Downstairs, my Linux server, the ReadyNAS NV+ storage server, a USB-attached hard drive, an Ethernet switch and the DSL modem are all connected to the UPS. Assuming I haven’t forgotten a vital piece of infrastructure, we should now be able to continue to use the Internet and make phone calls in the event of a power-cut.

All of the above equipment is good for a load of about 12% (the load fluctuates depending on whether I’m compiling code, whether the monitor’s turned on, etc.), which means that the UPS’s battery will provide back-up power for about 95 minutes. I can get an extra 15 minutes out of it if I turn off inessential equipment in the office.

I actually made a blunder when researching this product. It has a network management card, which is basically an Ethernet card with some bells and whistles, such as a port for attaching a temperature probe. I thought it additionally had a USB port for direct monitoring via a cable, but I somehow imagined that detail. Smaller units do have this feature, because they’re typically missing the Ethernet interface.

The only reason I wanted the ability to monitor the UPS over a USB cable was because that’s what the ReadyNAS box requires in order to be able to automatically shut down when the UPS’s battery is running low. Large disc arrays really don’t like it when the power just disappears and I wanted my box to be able to gracefully shut down.

Where there’s a will, there’s a way, though; well, sometimes, anyway. I set out to find a fix for the problem today.

It turns out that the ReadyNAS uses Network UPS Tools (or NUT for short) to perform its UPS monitoring. Reading NUT’s documentation, it became apparent that it has a number of back-end drivers for monitoring UPSes. One of these is snmp-ups, which, as the name suggests, uses the SNMP protocol to monitor UPSes.

My APC UPS can be queried and configured using SNMP, so this seemed like a good avenue to explore.

snmp-ups supports a number of MIBs. If you don’t know what a MIB is, think of it as a formalised set of questions that can be asked of a device. One of snmp-ups’s MIBs is called apcc, which is designed for use with APC units. Bingo.

Well, snmp-ups is an experimental driver and its man page warns that it’s not suitable for production use, so I wasn’t sure whether it would work. However, once I had installed NUT on my Linux server, configured it and started the server daemon, upsd, I was able to query for the status of the UPS from any host on the network.

The ReadyNAS has a feature that allows it to monitor a UPS attached to a second or subsequent ReadyNAS unit elsewhere on the network. Since those units are also running NUT, all the ReadyNAS is doing when it performs this remote monitoring is contacting the NUT server on the other unit. By installing a NUT server on my Linux box, I was hoping to pass it off as a second, UPS-backed ReadyNAS unit in the eyes of my ReadyNAS.

I went into the ReadyNAS’s FrontView interface, went to System -> Power -> UPS Configuration and gave it the IP address of my Linux server as the system to be monitored. Sadly, an error message popped up, informing me that a connection couldn’t be established.

I delved into my sysadmin toolkit and pulled out a perennial troubleshooting tool, tcpdump(8). This utility is used to analyse network traffic.

Within a few seconds, I was able to see where the communication was going awry: the ReadyNAS expected the system it was querying to have information available about a device called UPS. I had called my UPS apc in the NUT config.

Once I had renamed the monitored device and restarted upsd, the ReadyNAS was happy to believe that the NUT instance on my Linux server was, in fact, a second ReadyNAS box with which it could communicate.

Here’s the pudding containing the proof:

FrontView screenshot of ReadyNAS monitoring remote UPS via NUT server

I’m very happy I was able to get this to work, because having auto-shutdown on the ReadyNAS was the one thing that had prompted me to invest in a UPS in the first place. It would have been a bummer if precisely that feature hadn’t been available to me.

I also want my Linux server and desktop to automatically shut themselves down in the event of a power-cut. I could use NUT, but because the snmp-ups driver is marked experimental, I consider it safer to use the very mature apcupsd software. After all, apcupsd is designed specifically for use with APC UPSes.

apcupsd is actually a package I had on my list of things to investigate eight years ago when I was working at Linuxcare, but it’s taken me until now to actually delve into it. It’s very easy to configure and does exactly what it’s supposed to do.

There’s also a nice little GTK2 GUI client that will talk to an instance of apcupsd. It’s called gapcmon and is trivial to get up and running. You can put an icon for it in your tray and call it up whenever you want check the status of your UPS.

In conclusion, a UPS isn’t a very interesting device to add to your network. It’s expensive, heavy and you hopefully won’t get much use out of it. As is often the case with insurance, however, it can be very painful to discover after a catastrophe that one should have had it. For that reason, I feel good about my purchase.

Besides, this UPS has a few interesting aspects to it. For a start, it’s networked, which is always a desirable quality. It also works with Linux without the need for any proprietary software, which is another very desirable quality.

I hope that Netgear will add the snmp-ups driver to a future release of the ReadyNAS firmware. That will obviate the need for an extra computer to function as a NUT server.

Posted in Technology | 5 Comments