The Case of the Mysterious Missing MAC Address

Coriander is dead. Turmeric is alive. What does this mean? All the content on went away except for the PDFs and various other ebook formats which I was smart enough to grab off the RAID before it entirely keeled over. The uptime for coriander was ultimately 20 minutes tops before it keeled over. Not bad for a computer I built in 2003 and then rebuilt when Drexel had the Heat Wave of Death which caused me to request an extension on the finals. The box was two RAIDed 80GB IDE drives, running OpenSuSE I had installed as a desktop and later simply retired to serving up content to my Nook and XBOX.

Turmeric, however, is a first gen nVidia motherboard. And if this is people’s experience with nVidia, I am entirely, absolutely done with them as a motherboard maker. My network is pretty standard for home use. I have an honest to god Cisco router, a Cisco WAP, a comcast bullshit cable modem which is probably going to have a terrible accident so I can get one that works, and all the devices meander through those. The XBOX is UPnP permissions, nothing else does. Turmeric/Coriander had a MAC address reservation so they would come up, get the right IP, and then the cisco firewalls would pass traffic to them as needed. It worked swimmingly well until Turmeric refused to get the IP I had reserved for it. It would always get a different IP than reserved, but it would get the IP consistently. I racked my brain on this problem for a few hours and finally broke out ettercap to see WTF it was doing.

Turns out the first gen nVidia motherboards do something really stupid with DHCP. Actually lets rewind for a minute – they generally do really stupid stuff. This motherboard has hardware RAID also, but it only works for the SATA drives. IDE? Shit out of luck. To further add insult to injury, the bootp stuff for jumpstarting a box? Doesn’t work. Never figured that one out. Finally there’s only two default devices you get set in the BIOS. For the moment it’s CDROM and then the first drive in the RAID, but to actually do the install I had to change CDROM to USB after burning out an image to it. What the heck guys?

Now, I’ll save you the boring TCP spec – When the nVidia board comes up it actually sends a DHCP packet on its own which is nice. The problem here is the HLEN of the packet is… 0. Yup. Someone didn’t know what to put in the field, so they send 0. This causes the router (thank god) to respond to FF:FF:FF:FF:FF:FF, which while it’s not correct, works because it’s a broadcast packet. The adaptor (seems) to configure itself, then Linux does something goofy where it sees the adapter is configured, so it sends out a release/renew, which the router, apparently knowing the MAC address but having an entry for a bogus MAC, sends out a different IP (next in pool) for the correct MAC address.

The BIOS, of course, doesn’t have a way to disable this “convenience feature” and to add insult to injury, dmesg doesn’t work in Linux because the BIOS is doing something funky by itself. For right now I’m just ignoring it. But seriously nVidia, fix your shit.

Fedora release 16 (Verne) Upgrade Notes

I upgraded to Fedora 16. This laptop started out on FC14 and now is running FC16. I’ve managed to do it without a re-install.

Gnome is steadily improving, in a direction I am not interested in going. The UI continues to be unappealing, the lack of themes and configuration for keybindings (which continually have my hacks reset) makes Gnome worthless trash for a serious Linux user. The theming community produces the most hilarious amounts of bitching with comments similar to “Oh killer theme, how do I downgrade GNOME so I can use it?” Even the hacks don’t carry over. This continues on with GNOME into FC16 and it’s clearly GNOME’s problem.

KDE on the other hand, seems to get broken somewhere between FC16 and FC16’s upgrade to the newer 3.1 kernel. (3.1.0 versus 3.1.1). At least part of this is due to the intel graphics chipset in my laptop taking a huge crap on the kernel upgrade, so X didn’t come up correctly the first few times. The fix follows to be run as root:

  1. init 1
  2. rm -rf /tmp/*
  3. rm -rf /home/knarrj/.kde.*
  4. yum update intel\*
  5. yum reinstall kdebase


Start KDE in safe mode, disable compositing, let it settle. The stupid neopumk or whatever it’s called semantic desktop service will cause the machine to chug. Disable that trash since it can’t be uninstalled and your desktop will be a lot faster.

Post configuration crap – all your network settings will be gone. Your desktop theme will most likely be gone. However, if you got bit by the KDE not working correctly after upgrade bug where KIO kills itself, then your fix is here!

User Guide: How to Theme Gnome 3

If you’re running Ubuntu, you’re probably running Unity. Stop now and install a real distribution. Candidates include OpenSuSE and Fedora.

Alright now that we’re done that part, how do we fix the crap they shipped Fedora with? I know there’s potential in here somewhere, but it’s hard to find under the absolutely giant title bars on the windows. In fact, once you get used to pressing “logo” and typing something, it’s pretty cool. I still haven’t figured out how to unpin the favorites bar from the right hand side, but I’m sure it will come along. What I want to cover is how to install and use themes.

Step 1: Install gnome-tweak-tool. If you’re using fedora, it’s in yum.

Step 2: Find a theme you like. Places to check – Webupd8 or DeviantArt or More DeviantArt or Gnome Look. You’re looking for Gnome Shell themes (which change the notification bar and Activities pull out) and GTK 3 themes (which change the look and feel of apps).

Step 3: Grab and install ThemeSelector. Note that this is all beta, but it’s also the nicest version of a theme selector I have seen.

Step 4: GTK themes go in /usr/share/themes and gnome shell themes go in .themes (or also /usr/share themes). Pick one. I suggest /usr/share/themes since this is where legacy themes look for their resources. After adding a theme, restart gnome shell (ALT+F2, r).

My gnome shell is Ubuntu Ambiance Dark. My GTK theme is Hope. If you use gnome-tweak-tool, the window title bar is governed by the Windows tab’s current theme entry. The Interface tab will let you set the GTK+ theme. The Shell tab will let you set a Gnome Shell theme, but it seems to be less capable than the Theme Manager. The Theme Manager appears when you mouse top-left and select the themes tab. I know it’s more compatible than gnome-tweak since gnome-tweak throws out anything it doesn’t think is a theme, leading to much frustration.

HOWTO: Update You HP BIOS From Linux

Linux users consistently get the short end of the stick when it comes to BIOS updates. Oftentimes dmesg will show us that something is wrong in the BIOS and we have no way to fix it. This is especially infuriating when you have a laptop with no CDROM drive and it doesn’t like to boot off usb because of… a bug in the BIOS. This is the HP 6730b laptop with the 00D BIOS. It’s a disaster of a monumental scale. It’s a nice laptop when it works, but oftentimes with fedora or suse (or any other distro which uses the intel driver rather than vesa) the screen “flickers” and the laptop runs like junk until you CTRL+ALT+F2 it and back again a few times. The dmesg log will be filled with errors about intel_iommu and the kernel will report a crash.

Officially the problem is a buggy BIOS, what’s actually happening is the iommu register is being mapped to ‘0’ rather than any actual address. This means the CPU has it, the BIOS maps it incorrectly and then tells the OS to use ‘0’ as the address in memory. If the kernel didn’t do sanity checking your box would crash. As it stands it simply causes an interrupt (the screen flashing) and then it’s caught.

HP’s site is amazingly bad and offers up three options – Bootable CD, USB and floppy. The laptop won’t USB boot, because of the 00D BIOS. The laptop doesn’t have a floppy drive, and who does anymore? The laptop will boot off a CD, but this requires you to actually find a CD. I tried a DVD and booktyped it as a CD just for grins and it didn’t work. How are we supposed to update this crap? Install windows?

The workaround: If you don’t want to do this, the cheap workaround is to turn off “virtualization technology” in the BIOS and set intel_iommu=0 in the grub.conf. You will lose access to virtualization accelleration. It will still cause kernel problems, but it will happen much less frequently. This also fixes the problem with sleeping unless the box had the problem when it started up.

The fix: Ready to take the plunge? Get yourself an HP account first. Registration is free.

Install syslinux. In fedora, this is yum install syslinux and it puts in in /usr/share/syslinux. I would also install the perl stuff if you plan on doing a lot of servers with this. If HP continues this sort of crap for servers that they did with laptops, plan on doing this – a lot.

Grab sp50750. There’s actually an F17 BIOS, but it only affects Windows 7 and appears to have driver updates also.  If you want that one instead, use sp51317. As of right now the sp51317 download link doesn’t work, I believe they pulled it.

At this point I used wine to unpack the executable and I used rar to dump out ROM.CAB. Grab the ROM.BIN file and tuck it somewhere safe.

Grab sp50751. This is the F16 BIOS but this is for a different laptop. Do not apply this BIOS to your laptop. You have been warned. I didn’t try to apply the BIOS from that link to this laptop.

When you unpack that one, you will notice an ISO directory. You want to go in to that directory and mount the ROM.ISO – mount -o loop rom.iso tmp will mount the iso to a tmp directory. Create tmp first if it doesn’t exist.

Inside of your tmp directory (which is really the iso), you will see a /boot directory. Inside of that directory is a release.img file. Now we’re getting somewhere! You can run file release.img and see that it’s actually a floppy disk. Sweet! Copy this floppy image out of the ISO.

Mount up the floppy image. mount -o loop release.img tmp will do it, just like the CD. The difference is we can read and write to the floppy image while we can’t do that to a CD image. Doing an ls into our floppy image shows a 68PDD.BIN file, which is the BIOS. How am I so sure? It’s the only BIN file in there. Copy your ROM.BIN file we extracted earlier over the 68PDD.BIN file. Hit yes to overwrite it and then unmount the image.


We can boot from it!

Copy release.img and /usr/share/syslinux/memdisk to /boot.

Edit /boot/grub/menu.lst and add the following:

title Floppy Disk Image
root (hd0,0)
kernel /memdisk
initrd /release.img

Reboot and select “Floppy Disk Image” from the grub menu, and it will boot into freeDOS and flash F16 to your laptop! Next time you boot you’ll have the new BIOS after it does some tests.


[    0.000000] WARNING: at drivers/pci/dmar.c:633 warn_invalid_dmar+0x7d/0x8d()
[    0.000000] Hardware name: HP Compaq 6730b (SERIAL NUMBER)
[    0.000000] Your BIOS is broken; DMAR reported at address 0!
[    0.000000] BIOS vendor: Hewlett-Packard; Ver: 68PDD Ver. F.16; Product Version: F.16

New F16 bugs! Good thing HP fixes them all!

How To Find the VMWare Workstation License in Linux

Howto find the VMWare workstation license on your linux box?

No-one seems to have this, which is really, really annoying because if you move VMWare to another desktop or you’re like me and you alternate between your desktop and laptop, you want to use your companies VMWare workstation license. WHERE COULD THIS BE? The knowledge base suggests using regedit.exe to find it!

Protip – Novell may have been sold, but it wasn’t sold to Microsoft.

Check in $HOME/.vmware/ and you’ll see a file like license-ws-70-e1-200904 (for VMWare 7 and yes I know I’m behind).

Inside this file should be the line Serial = “XXXXX-XXXXX-XXXXX-XXXXX-XXXXX”

Resist the urge to copy this file wholesale to the new host, the host hash is what VMWare uses to figure out where each VM was copied. If you copy the file whole, any VM copied from the other VMWare host won’t go through the reconfig process and things will get screwy.

Leaving Novell’s SuSE Linux Enterprise Server

I was trying to ruminate how to write this so it wasn’t an out and out hit piece on how badly Novell shot themselves when it comes to running Linux. Then I realized that this was the company who brought us netware, and they couldn’t ever get ahead of it. Once I realized this, I understood the fundamental truth of the issue: Novell has always stood in Microsofts shadow and this is why they never achieved greatness.

Netware always ran on top of DOS. They were inseparable even as OS2 ran on top of DOS. DOS wasn’t even particularly nice, but the selling point of DOS back then was that it wasn’t UNIX. And it wasn’t even that it wasn’t UNIX in the sense that UNIX was unpleasant to use – UNIX was a great write-once-run-anywhere example with POSIX (for the most part) – it was that it was so darned expensive. The rise of Linux has been documented ad infinitum on this blog and elsewhere. If you’re unclear about it, grab a copy of the absolutely great Revolution OS and watch it. It’s not as much about politics as The Cathedral and Baazar but more about the people involved and their motivation in the face of absolute commercial adversity.

Lets consider, for a moment, the state of a Linux company as a whole in the present. Windows 7/Windows Server 2008 (I’ll just call it “Win 7”) finally has threading and user separation which is actually worth something. Windows 7 scripting is still a crapshoot with power shell but it’s vastly better than it ever was. People are finally starting to take .Net seriously since the Win 7 threading stopped sucking. It’s still hideously expensive to run, but it’s got the critical advantage that people are generally familiar with it. I know most of my readership runs Linux, bear with me, I run it too. When we say that Linux has better, more robust filesystems, this is true. When we say Linux is typically faster, this is true. When we point to Linux and we say that it’s more secure, this is true. The problem here in the present for a Linux company is that Windows 7 is probably good enough for most people. MS has put something out which is so good it raises the bar, and people who were not terribly happy with Linux due to their vendor might take a moment and say “Well now performance is similar for my specific purpose, lets give it a try”. I’m looking at you, Novell, because your tech support sucks, and this is coming from someone who’s been running Linux for 10 years and saw how badly Caldera’s support sucked.

What happened to Caldera? Novell bought them. Sigh.

When the company I worked for decided to shove off Novell’s SuSE, this is exactly the reasoning. Novell’s support sucked, in turn performance was marginal but vastly poorer than their marketing material would have shown, I suggested going to RedHat and the company ultimately decided Windows 7 was “good enough”. There is now balkanization where various departments are spinning off their own IT groups who were happy and satisfied with SuSE, but these IT groups are running OpenSuSE and they’re not using the Novell proprietary services. They would be every bit as happy on RedHat or Fedora or Debian or Ubuntu… as they would with SuSE. To them it doesn’t matter what Linux they’re running so long as the old faithful chugs along and dishes up their applications. To them, Linux is “good enough”. To my group which has to do things like directory administration and file sharing, Novell was a serious problem on whatever OS we ran it on (including XP clients to eDirectory which often crashed or did weird stuff when java was updated) and the new problem was that it wasn’t even “good enough”, it was totally blown out of the water.

Lets take a step forward and get out my Penguin Crystal Ball. Novell’s in trouble because they’ve allied themselves as a partner with MS and touted their AD compatibility. The problem was they did this before Win 7 really got a foothold out there and now the Big Push from MS was Windows 7 as a server OS. Now Novell, once again, finds itself competing with MS in MS’s own ballpark. This is from a simple technology perspective, nevermind that Novell only recently fixed up eDirectory to AD support to make it robust. From a money perspective it’s a no brainer – the cost of the license + the cost of support is about what you would pay for a similar amount of performance from MS. I’m not going to say they lied here, but the performance numbers were definitely padded in my opinion and it only got worse once the virtualization craze hit because it was even slower. RedHat here is a great example of doing this correctly – the price is competitive and the numbers are correct, but more on the point RedHat understands that MS is the Big Dog in the neighborhood and RedHat’s claim to fame is that they serve as AD replica servers flawlessly. Now you have a MS product which is fully supported, but if you have a branch office that doesn’t need all the bells and whistles, you can throw a RedHat Enterprise Linux server out there for $100 and serve up a full replica of AD. You can’t even buy Windows 7 for $100. Redhat’s other great idea – It doesn’t care if you’re a Mac or an MS client. It can serve up the domain and filesystems wholly transparently. Try joining an AD domain with an Apple sometime, see how that works out for you.

Novell, as a company, I am fairly sure will persist. There’s a lot of people such as ourselves who have legacy applications which run on Netware but want some bridge to the future. There’s also a place for companies right now who do Linux distributions because Linux’s kernel is going through growing pains at the moment with regards to hardware and “kernel module loaders”. The question is – how long can they hold on with both Apple and MS going for two different market segments? Apple is quickly becoming the defacto desktop to run for people who think buying a new computer is the solution to computer problems. They made a great choice putting a pretty face on the good old UNIX workhorse and they weren’t so vain as to make broad sweeping changes to POSIX (looking at you, Novell) or hide the command line. It even runs Linux software almost 100% so it has a wealth of applications. Win 7 in this respect is too little, too late. However, on the server side, Windows 7 is just what it needed to be to compete with UNIX deployments. Java, threads, scripting, POSIXy stuff and great privilege separation are all there. If I were Novell, I would be doing some serious soul searching.

If I were looking for a new way to update my infrastructure, I’d probably give Windows 7/Windows 2008 a try and put RedHat into service as a performance enhancer for my new, shiny system.

Update: In case anyone is wondering “what do I do if novell tanks?” – you can install OES (the Novell enterprise software) on top of RedHat. It works perfectly with a bit of librarly versioning work, but it can be done and it does run correctly.