Samat Says Samat Jain's personal weblog

If you pay attention to Linux-related news, you may have heard that Microsoft has contributed code adding Hyper-V acceleration to the Linux kernel. This event is not something that falls outside of their corporate agenda (whether it falls out of their strategy, I’ll let Steve Balmer voice).

Hyper-V is Microsoft’s hypervisor, included with the server editions of Windows (somewhat similar to VMware Workstation or Sun’s VirtualBox). It lets you run other guest operating systems within the currently running one (called the host OS). Typically, virtualizing guest OSes is slow. To improve performance, rather than virtualizing everything, special drivers and software can be installed into the guest OS to make certain things faster (such as graphics, disk I/O, etc).

The popular Linux hypervisors (Xen, KVM, etc) don’t have special drivers like these for Windows, so they won’t be able to run Windows particularly quickly. With Microsoft’s contribution, Linux now will ship with built-in acceleration for Microsoft’s hypervisor, making Linux run that much faster. If you were an IT shop that simultaneously needed to maximize performance and run both Linux and Windows, would you:

1. Run an open-source Linux hypervisor, and virtualize Windows (slow)
2. Run Microsoft’s hypervisor, included with expensive Windows Server licenses, and virtualize Linux (fast)

The answer’s clear. Microsoft’s kernel contribution brings them good PR and satisfies real-world customer demands, while continuing to promote their agenda to make running Windows seem like the best choice. Smart move!

A couple years ago, I setup my first gigabit Ethernet network. I wanted to test just how fast it could go with the equipment I gave it (that is, the NICs, cabling, and switches it operated on). Gigabit Ethernet, theoretically, can operate at 1000 Mbit/sec. This translates to 119.209 MiB/sec, units your OS typically displays when doing downloads (1000 Mbit/sec / 8 / 2^20). How close is your network setup to that maximum? Copying files between PCs, while being a very “real world” test, will be limited by how fast your disks can read or write. A specialized tool is needed.

While many system benchmark suites include network testing tools, most are not easily separate from their suites, and are not easy to install and use.

Enter NetStrain. It’s a very simple C application for Linux and MacOS X designed to stress network connections. Unfortunately, it’s not included in most Linux distributions or MacOS X, so you need to download and compile it yourself.

After compiling, use is simple. One machine acts as a server, and another machine acts as a client. Start the server first with:

This starts a server using IPv4 networking on port 9999 (use a different port if you know this is in use; remember to pick one above 1024 if you’re not running as root). On your client machine, start the client connect to the server (assumed to be running on IP 192.168.1.2 and port 9999):

NetStrain will then try to send as much over your network connection as it can as long as the client is running. NetStrain is very spartan, so there are not a lot of options. In addition to sending, you may want to test receiving, as well simultaneously sending and receiving. Check NetStrain’s README for details.

Most likely, you will not get anything near 119.209 MiB/sec—but hopefully, you’ll get better speeds than a normal 100 Mbit connection to make everything worthwhile.

What if you want to make things faster (without buying newer, better hardware)? There are many parameters you can tune on your operating system’s networking stack. However, in most modern operating systems, most of them are already set, or are automatically configured (e.g. TCP window scaling). The one major tunable is something called MTU (Maximum Transmission Unit).

Data is transferred over Ethernet in packets; the MTU defines the size of those packets. A larger packet size means fewer packets are needed to send the same amount of data, reducing the amount of processing that needs to be done by your computer, switches, and routers. Your computer’s NIC, switches, and routers need to support large-size MTUs, a feature often advertised as “Ethernet jumbo frames.” Jeff Atwood wrote an article on the promise and perils of jumbo frames that you may want to read if you’re interested.

Today, Adobe released 64-bit Flash for Linux. Finally, I can waste time watching ugly, pixelated Internet video on my 64-bit Linux desktop and laptop, just like all of my 32-bit-confined brothers and sisters on the Internet! (Yes, I know about npviewer—let’s not go there.)

What’s really interesting is that this is Adobe’s first 64-bit release of Flash. That is, Linux users got it first, before users of Windows Vista x64 and and MacOS X. It probably does not mean anything, especially since Adobe has mentioned 64-bit flash will be released at the same time across platforms, but you can’t help but feel good inside.

Go download it now and remember to report good bugs.

Update: Some quick notes…

• The tarball provided on the labs website is not the conventional Adobe Flash installer–it just contains the plugin. To use the plugin, drop the .so file into your ~/.mozilla/plugins/ directory.
• Make sure to uninstall your npviewer-powered 32-bit Flash completely (disabling the plugin within Firefox is not enough). I personally uninstalled it from my system to prevent any conflict.

Something that has bothered me forever is that applications that use GNU aspell for spell checking kept marking my name as a misspelling (I’m looking at you, KMail). Most front-end applications don’t provide a way for you to add your own custom words.

Apparently, creating your own personal dictionary is ridiculous easy with aspell.

If your language is English, create a file in your home directory called ”.aspell.en.pws”:

The first line is a required header. Every subsequent line is a word you want to add to your dictionary. I can’t believe I’ve let this sit for so long. Because it’s a nice text file, syncing this file between machines to take your dictionary with you is trivially easy.

Drupal-powered websites can be put into an “offline mode.” This is much better than most alternatives (such as taking the web server offline), especially for search engines, as the message and HTTP status codes given to users and robots alike will tell them to patiently come back later.

I’ve found that putting the site into offline mode makes database backups go much faster on heavily trafficked sites (which is obvious). However, for a particular site I was working with, this needed to be done in an automated manner, and on a dedicated database server that did not have access to the Drupal installation.

Most people take their Drupal sites offline through Drupal’s web-based administration interface. They can also be put offline through the Drupal Shell. Neither were suitable for me: the former cannot be automated easily, and the latter requires access to the Drupal installation. Fortunately, Drupal sites can easily be taken offline by setting things in the database, which can easily be done via bash scripts and the command-line MySQL client.

Given your database user is my_db_user, password my_password, and database my_drupal_db, the backup script would look something similar to:

Update: The original version of this article had some problems on some setups with the variables table being cached. I added another SQL statement to make sure this cache is flushed so the site actually reflects its configuration.

Update: This method really doesn’t work that well, and the more I think about it, there isn’t a way to get around writing something that interacts with Drupal. I’m working on a script that will be more fool-proof.

There’s been a bit of turmoil in the Amarok and KDE communities the past week with Amarok’s decision to only support MySQL Embedded in Amarok 2. Jeff Mitchell has written about the Amarok design decisions made.

I’m a little bothered by this, as it forgeos all the “semantic desktop” work that has gone into KDE 4, namely what’s provided by the Strigi and Nepomuk libraries. One thing the whole semantic desktop concept entails is that other applications will be able use data another application stored, but without care to what that other application was or how it was stored. For example, I should be able to share the list of all tracks in my music library, how many times I’ve played tracks, what tracks I think are my favorite, etc across music players. This kind of abstraction is, obviously, good for users, but bad for developers of proprietary software. They don’t want you to easily switch between applications that they do not control. Amarok switching to it’s own database store is a stab at this kind of desktop interoperability. I’ve my own thoughts to add, though, that support what the developers are doing…

Amarok is an awesome application. Dare I say, it’s a killer application on Linux—on several occasions this past year I’ve recommended people install Linux just so that they could play with Amarok and see how much better it is compared to what they were using (yes, I’m looking at you, iTunes).

Before Amarok, I used Music Player Daemon (mpd). I stopped using it after a while: the playlist management wasn’t very good; it would eat those playlists that I spent a lot of effort to make; the GUIs available at the time were lacking; and it was very slow when working with tens of thousands of songs. Some of this may have changed but I’ve not been motivated to look back.

Enter Amarok: I switched because the playlist management was so much better. I setup a MySQL server on my workstation to store metadata, as SQLite was much too slow. Amarok backed with MySQL is very fast—I dare others to find a library-based music manager that is faster with the number of songs I’ve thrown at it.

Balancing desktop interoperability with performance is a delicate balancing act. Interoperability is the hot thing these days—look at how Apple’s line of integrated software and hardware continue to sip market share from the Microsoft-powered desktop. But when it comes down to it, performance and other more perceived benefits are going to win out over desktop interoperability. The Amarok developers’ decision to go with MySQL embedded is a good one that will hopefully keep people moving to Amarok over proprietary alternatives.

SpamAssassin is one of the most-used spam filtering systems in use today. Unfortunately, because there are so many different ways SpamAssassin can be used, SpamAssassin remains subject to many performance problems. Fortunately, there are several speed-ups and optimizations that can be applied to most SpamAssassin installations to speed up its rule processing, especially on Debian and Ubuntu GNU/Linux-based systems. These instructions can be adopted to other operating systems as well.

This article does not discuss configuring your mail filtering system (i.e. procmail, maildrop). This depends completely on your setup, and more than likely there are plenty of other articles that describe the best way to setup what you want.

Dell Ideastorm is a Dell-run website where users can post and vote (anonymously) for ideas for Dell products. What’s the number one thing people want…?

People want Linux on their Dell machines. Remember to click the “promote” icon and make your vote count.

This past Ubucon’s key signing party was my first key signing party. One thing I noticed–signing keys after a key signing party is a boring repetitive task. Summarized from the Ubuntu wiki entry on typical key signing post-party protocol:

1. Retrieve all public keys of key signing party participants, using gpg –-recv-key
2. Compare the hardcopy fingerprint from the keysigning party to the fingerprint of the retrieved public keys, using gpg –-fingerprint
3. Sign the key, using gpg –-sign Send the signed key back, either by
   • E-mail: export the key, then e-mail it to the key owner, using gpg –-export -a | mail -s “Your signed key” user@example.com
   • Key server: send the key to a public keyserver, using gpg –send-keys

This is incredibly monotonous—and people have to wonder why Web of Trust-based encryption is not more popular?

The Debian signing-party package provides the utility caff to automate some of this. It’s not very friendly to “desktop” users, however:

• it’s a CLI application
• it requires a local MTA (/usr/sbin/sendmail in particular), or an “open” SMTP server, with no support for authenticated SMTP or SMTP/SSL
• the configuration file syntax is Perl and confusing; there are also few examples on the Internet

You could add authenticated SMTP or SMTP/SSL support to the script, but having to know how to hack Perl definitely disqualifies caffe from being a desktop-friendly application.

So, I hacked together my own key signing party script in Python that would send signed keys back to people via KMail. To use it, create a text file listing all key IDs you wish to sign, one per line. Pipe the contents of this list into the script:

The script will download each key, ask you to verify the fingerprint, and then sign it. It then will open a KMail composer window, pre-filled with the key owner’s e-mail address, a friendly template message (customizable in the script), and attached key. Review each e-mail to make sure it is kosher, and click send. Other than continuing to be a CLI program, I think this is much friendlier–the only manual work done is the creation of list of keys to sign, comparing fingerprints (this could be automated, but it seems in the spirit of the Web of Trust-based systems not to), and clicking send in a familiar desktop e-mail client.

Now for some notes…

It uses the DCOP automation features of KDE’s Kmail to send messages. You could similarly use Evolution and D-Bus, but I don’t use Evolution so I can’t contribute that bit of functionality. Mozilla’s Thunderbird unfortunately does not yet support any kind of automation features (as far as I know, anyway), so you’re completely out of luck if you use it.

DCOP with Python is a complete, utter, pain. The easy way to drag-and-drop boiler-plate code with kdcop did not work, as it appears the APIs have changed. A problem with KDE/Python dcopext’s module and multiple identically-named-functions sealed the deal for me and I gave up trying to use DCOP with Python, and instead settled for a hack of using the shell instead. I’m looking forward the one Linux desktop IPC protocol to rule them all, D-Bus, to debut in KDE4.

My script does not provide all the functionality of caffe. It, for example, does not encrypt the messages and their keys back to their owners. There doesn’t appear to be an easy way to do this with KMail and DCOP, so it’s a feature that will have to wait.

Fredico M Quintero pointed out some serious flaws in GNOME’s font configuration dialog; the Novell Product Design wiki also describes some problems. In a sentence that fits in with what I believe is GNOME’s “simplicity mantra”, GNOME should just get rid of its useless, confusing fonts configuration dialog.

Why does it have a font configuration dialog anyway? Well, unfortunately, GNOME’s setting daemon completely ignores several fontconfig settings and instead uses its own settings for things like antialiasing type, whether hinting is used, DPI, etc. You need the font configuration dialog to change these settings, or you have to dig through gconf. Most of this was put in place probably to subvert a broken X setup; instead of implementing these hack-ish workarounds GNOME should instead push to fix X instead.

It’s extremely difficult to discern the difference between the different types of antialiasing. GNOME’s dialog doesn’t let you select arbitrary text, or let you render text in-place so that you can quickly compare between different antialiasing styles and subpixel orderings. These settings, along with DPI, are unlike the rest of the settings in the font configuration dialog because they don’t apply immediately. They only affect newly started applications, and the dialog does nothing to alert you of this.

Do users really need to be able to select subpixel ordering from a dialog? There are very few LCD monitors that do not use an RGB subpixel ordering. The very few people who rotate their LCD monitors into portrait mode (including me, see my past article Misery with online reading of PDFs and the need for portrait monitors) would use VRGB. Why not just set RGB subpixel ordering if the user is using an LCD? VRGB if their display is rotated? Again, these are things GNOME could discover by querying X…

Lastly, do users need to change the fonts used by their UI in the first place? The majority of Windows and MacOS X users don’t deviate from the defaults at all—why would GNOME users be given a choice through this confusing dialog? GNOME instead should use the fontconfig aliases “Sans”, “Sans Serif”, and “Monospace” rather than letting users choose fonts. A fresh GNOME setup already uses these aliases as the defaults anyway.

Of the settings in the font configuration dialog users may actually want to set, whether to use antialiasing or not is the only one that sticks out to me as needing an option. I think that the dialog could be replaced with a simple, descriptive checkbox somewhere that read “Antialias text” that would toggle all the heuristics I’ve described above.