Wednesday, December 14, 2011

Manage System Startup and Boot Processes on Linux with Upstart

Want to start, stop, and manage services on your Linux box? Then you need to familiarize yourself with Upstart and take control of your startup and boot processes on Linux.
When Linux boots up, the first process that runs is called init. From there, init takes the task of starting up system processes. But which init? Turns out, there are several flavors of init, and it depends on which Linux distribution you're using and how modern the release is. Let's take a look at some of the backstory.

BSD, System V, Upstart and systemd init, Oh My...

You might think that the way that the system starts would be fairly well-agreed upon between Linux distributions. Unfortunately, that's not been the case and may not be the case for some time.

Linux isn't directly derived from UNIX, but it takes a lot of ideas from UNIX, including the way that the system starts. Some Linux distros, most notably Slackware, used the BSD-style init. Slackware (and its derivatives) is the lone holdout for this style these days, and it includes a System V init compatibility workaround so that applications that expect to support System V init scripts can install them normally.
System V (from UNIX System V), the precursor to was the standard for most Linux distros for many years. Distros using SysV init would have several directories under /etc such as rc0.d, rc1.d, through rc6.d for each runlevel — though many of the runlevels are not really implemented.
What's a runlevel, you ask? Essentially this is the grouping of services that are run by init. For example, runlevel 0 is "halt," runlevel 1 or S are the "single user mode," and runlevel 6 is reboot. On Fedora/Red Hat systems, runlevel 3 is a full set of services to run a multiuser system (networking, etc.), and runlevel 5 includes multiuser services and X11 with the display manager. You'll notice I didn't mention 2 or 4, because those runlevels are rarely used and runlevel 4 isn't even defined by default.
Note that you can switch between runlevels using the telinit command, like so: telinit 3 would switch to runlevel 3, and telinit 6 would tell the system to reboot. Note that you would usually want to just use reboot instead to reboot, but telinit 6 would also work.
Under each of the directories, you'd have a set of symbolic links that point back to scripts that include startup and shutdown directives for services like Apache and the SSH daemon. Each link has a name like KNNsshd or SNNcups, where the NN determines the order in which services are killed (K) or started (S). This became tricky with laptops and mobile systems, because the state of a computer might determine the order in which services should be started. See the rationale for Upstart by Scott James Remnant for more detail — but the short of it is that the SysV style init that served Linux well for many years became rather fragile and creaky when Linux adapted to use as a notebook OS as much as a server OS.
So now we have, or had, Upstart as the emerging standard. For now, Upstart is the sort-of standard for several distros. Upstart is used by Ubuntu, current Fedora releases, and openSUSE 11.3 includes it as an optional package. But the Fedora folks are working on a replacement for Upstart called systemd, which is a "system and session manager" that is meant to be a "drop-in replacement for sysvinit." There's a very detailed discussion of the rationale behind systemd by Lennart Poettering that is an interesting read if you're interested in the deep details. So at some point, you'll need to know Upstart or systemd, or both if you are managing systems using Upstart and systemd.
For now, we're going to look at Upstart.

Managing Startup Services

Now, when we're talking about startup services, I mean the system-wide services like SSH and Apache, not the applications started up by your desktop. That's a whole different kettle of fish, and depends on the desktop that you're using. For example, if you're running a recent Linux distro with GNOME, you'll look at System -> Preferences -> Startup Applications.
For applications managed with Upstart, you'll first want to look at the initctl command. This allows you to work with Upstart's init daemon. Naturally, you're going to need to use sudo to run initctl or be logged in as root.
To see what's running on your system, use initctl list. You'll see something like this:

alsa-mixer-save stop/waiting
avahi-daemon start/running, process 690
mountall-net stop/waiting
rc stop/waiting
rsyslog start/running, process 482
screen-cleanup stop/waiting
tty4 start/running, process 859
udev start/running, process 334
upstart-udev-bridge start/running, process 304
ureadahead-other stop/waiting
This is from a system running Ubuntu 10.10, you'll see a different list of jobs on Fedora 14 or another distro. The first field is the name of the job — like rsyslog. The second field is the "goal" of the job, followed by the state of the job.
To stop a job that's running, use initctl stop job. To start a job, run initctl start job. You can also request status of jobs with initctl status job. Finally you can restart or reload jobs with the restart and reload commands. What's the difference between restarting and reloading? Restarting does what it says on the tin — stops the job and then restarts it. The reload command sends the SIGHUP signal to the job, which can be used to tell a daemon to re-read its configuration.
For jobs that are managed by Upstart, you can edit their configuration file under /etc/init. They will have a few lines that describe which runlevels to start/stop on like so:

start on runlevel [2345]
stop on runlevel [!2345] 
Change the runlevels as necesary.
If you're using a desktop system, you'll want to install the Boot-up Manager — a GUI tool for handling runlevel configuration. The package is bum, and it will let you manage jobs and configure services by runlevel. It's simple to use, and shouldn't require a lot of guidance.
Note that Fedora uses Upstart in SysV compatibility mode, which means that a lot of jobs on Fedora are not managed with native Upstart scripts. For those jobs you'll want to use the service command.
To see the status of all jobs on Fedora using service run:
service --status-all
This will show which jobs are stopped, running, and in a few instances much more information. For example, iptables will spit out its entire set of policies that are running at the moment.
To manage a job with service use service servicename command. The servicename is the name of the configuration script under /etc/init.d. For example, to start and stop the openSSH daemon, you'll use service sshd start or service sshd stop.
If you want to tweak the services by runlevel, use the chkconfig command. You'd use something like chkconfig --level 35 cups on to tell chkconfig to make sure that cups is started in runlevels 3 and 5.
What if you want to add a service or modify the scripts for existing services? One quick and dirty way is to use /etc/rc.local. The script should be run during any startup or change of runlevel. Another way is to modify the scripts under /etc/init.d (for SysV compatibility) or /etc/init (native Upstart scripts) or write your own for a new service.
Typically, this is unnecessary — almost any service that you'll be using should have init scripts pre-written. Writing an init script, whether for Upstart or for SysV, is a bit outside the scope of this tutorial.
Since many users are going to be transitioning to systemd, we'll take a longer look at managing jobs with systemd when Fedora 15 comes out.
Most of the time, your interaction with managing services should be minimal unless you're working with Linux as a full-time admin. If you are doing system administration, or planning to, you should already be familiar with the init system that your distros are using. If not, now's a good time to learn. If you're a desktop Linux user, you probably won't be managing services very often, but it's a good to know the basics that we've covered here in case you need to do any troubleshooting or just want to fine-tune your system.



No comments: