Sunday, January 14, 2007

Solaris on Iomega p405u

Some time ago, I was looking for a NAS device to increase the storage on my home network. The prosumer offerings at the time were a bit lacking, and the price was a bit too high for my tastes. Most of them used an embedded SBC running a proprietary OS of some sort, and were mainly catering to Windows/CIFS sharing. The also generally only included a single NIC, and a single or dual drives. I wanted to have a something with a minimum of RAID5, and preferably 4 ATA drives. I settled on the Iomega p405u.

The p405u is a standard PC for all intents and purposes. It contains an Asus TP20 motherboard with 2 Intel 10/100 NICs, 2 onboard IDE channels, a HighPoint 370 RAID card. In this particular configuration, it's outfitted with a half a gigabyte of RAM and a 1.0GHz PIII. There are 4 hot-plug ATA drives in caddies in the front, mine are currently 80gb. From the factory, it ran a stripped down version of FreeBSD with a Web UI tacked on. It works, and I used
it like this for some time.

However, I decided it was time to replace the built-in OS with something a bit more functional. I want to upgrade the drives and I was unsure of how the p405u software would handle that, plus there are just better things out there. I couldn't even SSH into the appliance as it was, so that Iomega software had to go. You won't find a floppy or a CD-ROM on this sucker, or even a USB port, so I set up my PXE environment to boot a couple of alternative OS distributions catered to storage. First was FreeNAS, which is based on m0n0wall. I've run m0n0wall since the first release for the Soekris 4501, and I've been very pleased with it. After reading, though, it seemed that FreeNAS is lagging a bit in development, and that it would require a lot of work just to get it PXE booted. Since I have no idea if the hardware will even work with their kernel, I decided to skip it and move on to OpenFiler. OpenFiler aspires to be great things, but it can't escape the fact that it's running Linux and LVM underneath, and I found it a pain to set up and administer volumes and shares.

I set up Solaris 6/06 to PXE boot on the system and installed it. It was able to find all of the devices in the system, except for the 10/100/1000 NIC on the PCI riser card and the HighPoint 370 RAID controller. This presents a problem, since I can't see the other 2 drives in the NAS, and it makes it somewhat limited for storage. After some searching, I found that you can override Solaris' pci-ide driver and have it bind to a device it doesn't detect. To test, simply add the following to the GRUB kernel stanza:

-B pci-ide="pci1103,4"

The 1103,4 comes from a listing of `prtconf -pv`. The relevant sections here are the vendor-id (00001103) and the device-id (00000004). If you wish to make this change permanent, you can run:

# eeprom pci-ide="pci1103,4"

Once booted with this flag, the system hung after detecting the first drive. After a post on the OpenSolaris forums, it was suggested to me that this could be the result of bugid 6414472, which is not currently fixed. I started to build a new kernel using the OpenSolaris sources, but the amazing OpenSolaris team had a better solution. Simply boot with '-kd' to drop into kmdb before booting the OS. Then, run the following:

::bp ata`ata_id_common
:c
::delete 1
ata_id_common+0x39?w a6a
:c

Yes, that's right, we've just changed the value in the debugger, and it worked beautifully. To make this change persistent across reboots:

# adb -w /platform/i86pc/kernel/drv/ata
ata_id_common+0x39?w a6a
$q

My hat goes off to the amazing OpenSolaris team, this is by far the most amazing fix I've ever done to a system. Bravo!