Site Move!

After almost two years of silence, I've moved this blog to Blogger and am posting again. There's a lot of updates to make but for the time being I'll just mention that I've moved all of my infrastructure to Google Apps away from the servers in my basement. I'm using EditDNS for DNS as well. I'm still shopping for a hosting provider for the couple of other sites I used to manage, but I'm debating between HostGator and a few others. I can't guarantee that I'll update this site any more than I did before, but at least it's easier to post updates through Blogger than it was navigating the ridiculously complicated blogging product I had on Plone.

Updates

A.K.A. Why haven't I updated the site?

I've been super-busy the past several months, I started a new job, bought a house, Abigail started school, and I started back to MTSU. So, needless to say, I haven't been posting on the site much.

After the move, I got rid of a bunch of my collection in various ways. I'm down to just a few systems now, which certainly helps the storage problems. I still have the SS1000E, but it is super-clunky and only runs Solaris 8, though it runs it pretty well. I'm thinking about replacing it with a E4500, which will stack nicely on top of the SSAs I have.

I found a great deal for someone wanting a cluster at home. First-generation blade chassis are super-cheap on eBay, I say a 20-blade PIII 1GHz chassis for less than $1k. The switching is built in, as well, and is usually GigE.

I got a great deal on a SAN for the house, 6 StorEdge A5200s, 4 Fibre switches, and a StorEdge rack.

I also found a great deal on cheap 1U servers: Netra T1 200s and Netra X1s. The X1 is a dual IDE drive system, has a short depth, 2 10/100 interfaces and a 400/500MHz USIIe. The T1 is almost the same, except it takes 2 SCA drives.

That's about it. Maybe if I do anything else cool I'll update the site, but there hasn't been too much going on. New toys = new updates. :)

My 3 Parallel Systems...

The way I look at things, Parallel computing falls into three basic areas:

• Shared Memory Systems (SMP)


• Distributed Systems (Clusters)


• Hybrids (NUMA)

I plan to have one of each running at my house, but I got the first one up this week. I wanted something that was fairly massively multi-processor, and I just so happened to have a bunch of Sun parts in the garage and a spare SparcServer 1000E. I loaded it up with 4x 4.5GB drives, 8x 85MHz SuperSparcs, and 2GB RAM. After a failed attempt at SMP Linux, I've installed Solaris 8. So far so good. Now, to get pkgsrc running...

Setting up a VMware HA Cluster using RedHat Cluster

Once upon a time a company I worked for was migrating their systems to VMware in a consolidation effort. They had a SAN, they had a BladeCenter, and they had VMware. And we were all pretty pumped about the VMware, let me tell you. It was like the second coming.

But there was a problem, you see. We were using RedHat Cluster on Debian Sarge. And RedHat Cluster didn't support fencing machines through VMware. What's fencing, you ask? Well, in cluster-land, when a node becomes unreachable or out of sync with the others, it is 'fenced'. This is a mechanism to ensure data integrity by forcably removing the node from the cluster and the cluster storage. In RedHat Cluster, this can be achieved by either removing the node from the storage in the switch, or by rebooting the node via a LOM card or a addressable power supply. If we did either of those in VMware, we'd take everyone else down with us, and that would be very, very bad.

Well luckily for us, VMware provides a Perl API to use to connect to the host machine. I used this API to modify the fence_apc script, which is used to telnet to an APC Masterswitch and fence a node, to a version that would work for VMware Server or ESX.

First you must set up a fence.ccs file.

fence_devices {
  esxblade1 {
    agent = "fence_apc"
    ipaddr = "10.0.0.1"
    login = "fence"
    passwd = "password"
  }
  esxblade2 {
    agent = "fence_apc"
    ipaddr = "10.0.0.2"
    login = "fence"
    passwd = "password"
  }
}

This defines two fence devices, esxblade1 and esxblade2. It lists their IP addresses, a login ID and a password, and the agent to use, which in this case is fence_apc. I placed my script in /sbin/fence_apc and moved the original out of the way to make sure there wouldn't be a problem. The login ID and password should be an account on the VMware host system that has access to reboot, startup, and shutdown the VMs in question.

Next, we create a nodes.ccs file.

nodes {
  node1 {
    ip_interfaces {
      eth0 = "10.1.0.1"
    }
    fence {
      vmware {
        esxblade2 {
          port = "/home/vmware/node1.vmx"
        }
      }
    }
  }
  node2 {
    ip_interfaces {
      eth0 = "10.1.0.2"
    }
    fence {
      vmware {
        esxblade1 {
          port = "/home/vmware/node2.vmx"
        }
      }
    }
  }
}

This specifies the public interfaces for two machines, known as node1 and node2. It also defines a fencing mechanism for node1 and node2, vmware. The vmware fence script takes as its argument a 'port', which is really the path on the host system's filesystem to the .vmx file for the virtual machine.

The script for fence_vmware is available here.

You should be able to use this, in conjunction with the documentation for RedHat Cluster, to set yourself up a test VMware cluster. If not, feel free to shoot me an email and I'll try and help you out.

Sun announces partnership with Intel

Sun and Intel announced today that Sun will begin shipping Intel-based servers and workstations in Q1 2007. This is pretty exciting, Intel has some nice looking stuff coming down the pipe and I'm glad to see that Sun isn't tying itself too closely to a single CPU vendor, unlike some *cough* companies.

The 'official' announcement is available here, and Jonathan's blog posting is here.

Mac OS X... it will remind you of Unix.

OS X may be the greatest Unix GUI system ever invented, but it still leaves me irked at times.

A friend of mine asked me a few weeks back about a way on OS X to redirect sound from one system to another. I proposed this solution, thinking it rather clever:

dd if=/dev/audio | ssh remotehost dd bs=1k of=/dev/audio

Seems like it should work, right? Wrong. For you see, there's no /dev/audio on OS X. Or /dev/mixer. Or anything, really. How annoying. So how does one use audio in a standard Unix-y way on OS X?

I had another friend ask me about making a frequency generator application on her Mac to show her Physics students. In the 'old days', this would have been no trouble, but it seems times have changed. It's now insanely difficult to output any sort of sound on any computer, much less a Mac. Since there's no /dev/audio device, none of the examples I found would work. Luckily for her, I found an application that should do the trick, but I'm still upset that I couldn't successfully make it work using Python.

If anyone's interested, the link for the program I found, called AudioTest, is located here. It looks to be pretty cool.

Setting up an Enterprise Home Network Part 2

I'm a System Administrator. A lot of times I wonder how it was exactly that I got to where I am. You can't really learn to do what I do from a book, or a series of books, or from a school. And yet, I have read tons of books and spent a major portion of my life in school. And all of that has helped tremendously, but it wouldn't have been enough.

When I got my first admin job, it was because I knew a thing or two about Linux. I had been running Linux since the early 90's, and had several machines set up at my house running various things. That's really what started it all. I was just playing. Years and years of playing and doing stupid things at my house that I had no reason to be doing, but I did anyhow. I made my hobby of computers relevant to the real world.

So, to help others who might be interested in doing the same, I'll put some advice I give pretty often to people who are wondering about what hardware to buy. This will be a multi-part series, with the first part dealing mainly with the Ethernet network infrastructure.

Wireless / Router

Everone has wireless already. It's usually provided by a small black and purple box made by Linksys, which does triple-duty as a switch and a NAT router. Real world relevance: 0.

If you want something a bit more serious, you could use what I've used for the past several years, a Soekris box running m0n0wall. It's a nice little system that has worked flawlessly, and can expose you to a variety of higher-level networking things that your Linksys just won't do. People to run these systems in production as well, and I can see why. But, it's not what you're likely to encounter if you walk on a job where they need a firewall worked on.

If you really want something to get you familiar, I'd recommend a Cisco PIX 501. They're relatively inexpensive, easy to find on eBay, and run the same OS that the large grade Cisco firewalls do, like the ASA. Of course, the ASA doesn't have any sort of wireless support, so you might want to pick up a Cisco Aironet 1100, as well. The PIX will give you VPN access, IDS, URL Filtering, and hardwaare Failover. The Aironet device is generally found with an 802.11b card, but is upgradable to G and other standards.

Switches

When I needed a new switch for my office, I picked up a Cisco Catalyst 5000. It was about $100 on eBay, and had 2x 24-port 10/100 blades and a single 48-port 10bT blade. You might not need something this crazy, but it has its advantages. For one, you can configure EtherChannel (or 802.3ad aggregation) between the blades and increase a single host's bandwidth to the switch. Also, you can create 802.1q VLANs to make your large switch investment into the last switch you'll ever need to buy. If you want something smaller, pick up something from the Cisco 2900 series, like the 2924 or the 2950. The 2924 won't run the latest IOS and won't do SSH, but it's adequate and will do VLANs and aggregation for you. The 2950 is just a continuation of the 2924, and is a bit more recent.

With all of this equipment, you can segment your network in a much more palatable way than usual. You know how every time you have a friend over that wants to use your wireless network, you have to give them your key to access it? No need to do that anymore. You can create a separate VLAN for wireless guests, on a separate wireless network, and leave it wide open. Restrict it to web access only, and deny it access to any of your internal machines.

Perhaps you want to run some externally facing servers, but you're concerned that they might get hacked and access your sensitive internal machines. No problem, just create a DMZ VLAN, assign your servers into it, and set up the PIX to use a 1:1 NAT to those systems on a port-by-port basis.

Hey, look at that, your entire network is Cisco now, more secure, and you're learning! Get yourself an IOS router of some description, and you can train at home for your CCNA. Stay tuned for the next part, where I will discuss Storage for your home network.