Fixing Ethernet Device Creep

In CentOS, udev remembers various things about your network adapters from boot to boot, most importantly, the MAC address. The reasoning is simple: Boot once, get it setup, shut it down, add a second NIC, boot back up, now which one is supposed to have which address? Generally the idea is this, udev keeps the adapter MAC associated with the device name (so ::08 get's eth0, and ::2b gets eth1. remove ::08 and you'll still have eth1). Then the network scripts in /etc/sysconfig/network-scripts/ named ifcfg- are read by /etc/init.d/network on start up and it associates eth0 -> IP1 eth1 -> IP2.

Problem is, say you're not really set on the MAC you've got (it's a VM and you cloned it, clones get a new MAC. It's a physical box and that card died, replace it and it's got a new MAC). How do you get that coveted eth0 label back so you don't have to change a lot of things**? Well, to quote Dr. Seuss "We can do it, we know how".

Pull yourself a quick ifconfig listing. Note the MAC address of the NIC you want to be, say, eth0 in this case.

[root@mysystem ~] # ifconfig |grep HWaddr
eth1 Link encap:Ethernet HWaddr 00:50:56:88:00:74
eth2 Link encap:Ethernet HWaddr 00:50:56:88:00:75
eth3 Link encap:Ethernet HWaddr 00:50:56:88:00:7b

In this case, there used to be an eth0 and when I cloned this box, I got eth3. Let's change that.

[root@mysystem ~] # cat /etc/udev/rules.d/70-persistent-net.rules
# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:50:56:88:00:74", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:50:56:88:00:75", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:50:56:88:00:7b", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

What we need to do is edit this file and change that last 'NAME="eth3"' to 'NAME="eth0"'.

Now, I'd say reboot, but you need to do one more check first... Make sure you have something in /etc/sysconfig/network-scripts/ifcfg-eth0 and make sure of two things, if you have a Device= line that it says eth0 (In case you thought you'd be clever by copying ifcfg-eth3 to ifcfg-eth0 and rolling with it). Also, make sure if it has a HWADDR= line that it equals the new MAC and not the old one. Truthfully, since udev does the association from MAC->device name, this line isn't necessary and I almost always comment it out (# at the front of the line works in this file), but CentOS's "system-config-network" script adds it in there, so watch out. Okay, now reboot!

** I know, this doesn't seem like that big of a deal, but if you have a complex setup of many NICs and/or are using a monitoring program to watch traffic on a specific NIC, you now have more to change. Especially if there's a group of servers that match and now this one doesn't, etc.

GeoIP in CentOS 6.x

Simple one today... So our nginx rpm requires GeoIP. It's useful, and its also been a standard part of CentOS for a bit now, but apparently it's not in CentOS 6. To get it, simply run

rpm -Uvh http://download.fedora.redhat.com/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm

This will add in the right repo to include it in your yum searches. Then you can just run

yum install geoip

and you'll be good to go!

SSLCertificateFile errors in Apache httpd-2.2.21

Apache 2.2.21 is out, so, being the 'on the ball' sys-admin I am, I pulled it down and rolled myself an rpm using rpm build (which I should probably make a post on later). After a few minutes of compiling, I had my spiffy rpm and I was ready to deploy it.
I first pushed it to all of our beta sites as they most closely matched our live servers and got them running in no time with a little 'rpm -Uvh httpd-2.2.21.rpm' command. After a week of that running well, I pushed it to our dev server this morning and ran into a problem... Where I previously had:

SSLEngine on
SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
SSLCertificateFile /etc/httpd/conf/ssl.crt/secure_mysite_com.crt
SSLCertificateKeyFile /etc/httpd/conf/ssl.key/secure_mysite_com.key

in our vhost config, it no longer allowed me to start httpd. Looked in the error log and I found:

[Tue Nov 08 09:38:46 2011] [error] Server should be SSL-aware but has no certificate configured [Hint: SSLCertificateFile] ((null):0)

Of course my first response was "But I DO!". I dug around and found many places where it was defined, always correct. After paring down our long list of vhosts a bit, I was able to get one to work. Apparently it had the following lines:

SSLEngine on
SSLCipherSuite HIGH:!SSLv2:!ADH:!aNULL:!eNULL:!NULL
SSLProtocol -ALL +SSLv3 +TLSv1
SSLCertificateFile /etc/httpd/conf/ssl.crt/secure_mysite_com.crt
SSLCACertificateFile /etc/httpd/conf/ssl.crt/intermediate_bundle.crt
SSLCertificateKeyFile /etc/httpd/conf/ssl.key/secure_mysite_com.key

I don't know why I had to explicitly define my CACert on top of the others, but this made it happy and I was able to go along my way!

I found out after rolling this to a few other servers that it's also hypersensitive to Vhosts that are on port 443 but don't have ssl configs. Make sure you have the above lines or an include file with them in every vhost you have configured to run on port 443!

Date Changes on XenServer VMs

If you've ever tried to run 'date -s' on a virtual machine running on a XenServer host, you'll have no doubt noticed that it returns the right date, only to be back to it's previous setting the second after. This has to do with the kernel level date sync between the XenServer Host and it's Virtual Machines. While this is very handy as it means that all servers have their date set by the hardware node without having to run ntp in all of the virtual machines in your setup (though running ntp between XenServer Hosts in a pool is recommended), it means you can't update a single vm's date to something other than that of the XenServer Host. Luckily, this is simple to fix!

To allow a single VM to have a unique clock, log into that VM and run:

echo 1 > /proc/sys/xen/independent_wallclock

And to change it back:

echo 0 > /proc/sys/xen/independent_wallclock

I've tested this on Citrix XenServer 5.5 and 5.6. If you're going to leave it like this for any decent length of time, I'd Highly recommend setting up an NTP client on the VM to keep thing's synced somewhere. If it's a completely weird time setting, yet based on real time, you can always have a script on a host with NTP run an ssh command hourly to set the time based on your offset.

Installing New Fonts in Gnome 3.x

I found a few suggestions on how to do this online. Browsing to fonts with Nautilus and adding plugins to the system config pages and I really just want to say that's all overkill. This is what I found; it's REALLY easy to add fonts to Gnome 3.

1. Open up a terminal, see if you have a ~/.fonts/ dir. If you don't make one.
2. Copy new fonts into this directory.
3. Congrats, you're done. Open a beer and assign some new fonts to things.

So far I've found .otf and .ttf fonts work well. .svg, .eot, and .woff don't, but there may be additional packages to make them work. So far I haven't needed to as most fonts seem to be found in the .ttf format anyway.

As a side note, I've found a lot of 100% free to use fonts (even commercially) at http://www.fontsquirrel.com I'm sure there are some other great sites out there too, but this one should get you started on your journey into font nirvana.

PHP 5.3 in Centos 5.x

So, CentOS doesn't come with php 5.3. I know what you're thinking "But there are rpms for it, I can see them listed in yum!" Okay, true, if you do a "yum search php53" you'll get back a list of php 5.3.x packages. But try to find something like memcache or mssql in there? Not there. However, IUS has the rpms you've been waiting for. All you need to do is (for x64 CentOS):

wget http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/ius-release-1.0-8.ius.el5.noarch.rpm

wget http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/epel-release-5-4.noarch.rpm

rpm -ivh epel-release-* ius-release-*

Hint: if you get a 404 on these links, open http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/ in a browser, find the 'ius-release' and 'epel-release' files and substitute them for these.

You'll then have the IUS repo installed so you can simply "yum install php53u". For me, this consisted of a list of php53u-??? files. If you want it all, yum is nice enough to oblige by running "yum install php53u-*".

It is important to note that these files do not necessarily obsolete CentOS Stock packages. You will most likely have to run "yum erase php" before you install the new php53u packages or risk overlap/inconsistency.

I know this isn't that much of a trick to accomplish, but I just had to do this and I wanted to write it down for myself since I have to do it on a number of hosts... Figured I'd share it here.

Welcome

I'm hoping to make this a technical blog about random issues I come across during my various technology related endeavors. I'm not sure if it will be useful to many people, or to me in the long run... I'm hoping, however, that not only will it be a good way to keep track of problems and solutions I come across, but a good way to share that with people other than myself who are, perhaps, running into similar difficulties. I may from time to time post an interesting link or an op-ed piece if I have time and/or a reason for such things. Thanks for reading!