Controlling runaway processes on Linux

Sometimes, buggy memory hogs can choke your machine. Here, two tricks: one to recover from a memory choke, another to prevent memory chokes forever.

Misbehaving application frozen?

Has an application stopped responding on your machine? Well, as long as your machine is still responsive, you can use these tricks to nuke it safely.

On KDE

Hit Ctrl+Alt+Esc. Your mouse cursor will change -- from an arrow to a small skull/crossbones combo. Hit the stubborn application with the skull.

It'll die.

On GNOME

Add a new launcher to your panel and set it so it executes the xkill application. Now, when an application starts stupidifying itself, just hit the launcher you created (the cursor will change to a square-type "target"), then hit the application with your mouse cursor.

It'll die.

Application choking your machine?

Of course, if your machine is already too slow to use these tricks, they won't help you much. Here's why, sometimes, your machine ditches itself into a molasses pit, and how to rescue it from certain death.

Memory and pathologies

You see, almost all applications have this pathological idea (encouraged by the operating system) that memory is a limitless resource -- and when they go overboard, the operating system just dips into the hard disk to simulate memory.

Sometimes, bugs in an application do cause them to go haywire, requesting memory like there's no end. Once your machine goes down that lane, there isn't a simple way to recover it, short of powering it off forcibly.

A fortress-like operating system like Linux isn't supposed to die, and yet it does. However, the fortress I'm so proud of can, and does, provide you with effective measures against premature deaths of these sort.

A last-resort trick to recover your machine from the brink of death

Have you heard of the magic SysRq key?

No?

Well, it's magic. It's directly shunted to the Linux kernel. You press ALT, press the PrintScreen (SysRq) key, and while holding them both down, press one of the letters (each letter has a different function assigned to it).

It's not normally enabled, but you can enable it by putting

kernel.sysrq = 1

in your machine's /etc/sysctl.conf file. Oh, and then rebooting.

Here's why it's useful.

So, what does SysRq do, really?

Suppose a GUI application you just opened is starting to swallow massive amounts of RAM. Like, one gigabyte, perhaps? Your machine is locking up, and you feel the mouse start to stutter at first, then freeze completely -- while the hard disk light in your computer's front panel is lighting up frantically, gasping for airmemory.

You now have three choices:

1. Sit it out and let the Linux kernel detect this situation and kill the abusive application. This can take way more than 15 minutes.
2. Press the computer's power off button for 5 seconds. This shuts your machine down uncleanly and leads to data loss.
3. Hit the magic SysRq combo: Alt+SysRq+K.

Should you choose option 3, the graphical subsystem dies immediately. That's because Alt+SysRq+K kills any application that holds the keyboard open -- and, you guessed it, the graphical subsystem is holding it open. This premature death of the GUI causes all GUI applications to die in a cascade, including the abusive application.

Two to ten seconds later, you will be presented with a login prompt.

Sure, you lost changes to all files you haven't saved, and all the tabs in your Web browser... but at least you didn't have to reboot uncleanly, did you?

But, Ctrl+Alt+Backspace?

Once the machine is in a critically heavy memory crunch, Ctrl+Alt+Backspace will take too much time to work, because the windowing system will be pressed for memory to even execute. The magic SysRq key has the luxury of not having that problem ;-) -- if Ctrl+Alt+Backspace were an IV drip, SysRq would be like a central line.

Why this key combination exists

The reason this key combo exists is simple. Alt+SysRq+K is called SAK (System Attention Key). It was designed back in the days of, um, yore, to kill all applications snooping on the keyboard -- so administrators wishing to log in could safely do so without anyone sniffing their passwords.

As a preventative security measure, it sure works against keyloggers and other malware that may be snooping on your keyboard, may I say. And it most definitely works against your run-of-the-mill temporary memory shortage ;-).

Advantages/disadvantages

Well, the major disadvantages are:

• Anyone with keyboard access can reboot or hang your machine using a SysRq key combination.
• Once you hit it, since the GUI dies, all of your open applications close, forcibly.

But, on a memory crunch, this beats rebooting hands-down. And that's the biggest advantage.

A definitive cure to runaway applications

Become an administrator (root) and use your favorite text editor to open the file /etc/security/limits.conf:

# /etc/security/limits.conf
#
#Each line describes a limit for a user in the form:
#
#            
#
#Where:
# can be:
#        - an user name
#        - a group name, with @group syntax
#        - the wildcard *, for default entry
#        - the wildcard %, can be also used with %group syntax,
#                 for maxlogin limit
#
# can have the two values:
#        - "soft" for enforcing the soft limits
#        - "hard" for enforcing hard limits
#
# can be one of the following:
#        - core - limits the core file size (KB)
#        - data - max data size (KB)
#        - fsize - maximum filesize (KB)
#        - memlock - max locked-in-memory address space (KB)
#        - nofile - max number of open files
#        - rss - max resident set size (KB)
#        - stack - max stack size (KB)
#        - cpu - max CPU time (MIN)
#        - nproc - max number of processes
#        - as - address space limit

Add the following line anywhere on the file:

*      soft      as      512000

You will need to restart any sessions (graphical/terminal) for this change to take effect. Additionally, you will need to restart the graphical session manager (GDM or KDM).

Of course, 512000 is just my favorite setting -- but that's because I have the privilege of using multi-gigabyte memory sticks on my machine. If you have much less memory than me, you will want to tune this, while keeping in mind that modern applications can and do take more than 300 MB under exceptional circumstances.

Like, for example, Firefox with 50 tabs/windows open. Or Evolution managing 2 GB of e-mail. Hey, both circumstances happen to me, but I guess I'm an oddball.

What this "magic incantation" in limits.conf does

limits.conf is the file that lets you set per-user/process/system resource limits. There are several limits to choose from.

One of them is the address space (as). The address space refers to the maximum amount of RAM (in kilobytes) that a process may request from the operating system. Any requests above the configured limit are simply refused.

Advantages/disadvantages

The fortunate side effect of this recipe is that the majority of applications will disappear and die a horrible death if they request memory indiscriminately. Which beats having to turn the machine off.

The unfortunate side effect of refused requests for more memory is that the majority of applications will disappear and die a horrible death if they request memory indiscriminately. That's because they don't know how to cope with more memory. Until software developers actually implement error handling for lack of memory, this will be a small nuisance -- hey, save often and you'll be safe ;-).

Temporarily disabling memory limits for picky applications

Another disadvantage: certain applications don't run when a limit is set. WINE is among those applications. However, you can use a terminal window to disable this limit temporarily (within any applications started from the terminal):

[rudd-o@andrea /media/windows/UnrealTournament]$ ulimit -v unlimited
[rudd-o@andrea /media/windows/UnrealTournament]$ wine System/UnrealTournament.exe

That's it for this tutorial

So, the infrequent dead application, SAKing your computer, or powering it off... what do you prefer? Me... well, once I upgraded my new "desktop" machine (1U Dell PowerEdge SC1425) to 2 GB, I haven't looked back.

But my old machine did thank me a lot for keeping it off memory crunches :-). It's now safe in PVR heaven, piratingtime-shifting TV shows for my pleasure.