Personal tools
You are here: Home Linux and free software Starting ZFS-FUSE up properly

Starting ZFS-FUSE up properly

Posted by Rudd-O at Jan 22, 2009 02:25 PM |

ZFS is the best filesystem ever invented. And FUSE brings it to Linux. There are, however, some caveats you need to keep in mind if you want to start it properly.

Here is a collection of tips you must pay attention to:

  1. Put the ZFS PID file in the root filesystem.
  2. Unset the LANG environment variable.  Failure to do so will cause ZFS-FUSE to hang if your /usr is on ZFS.
  3. Immunize ZFS-FUSE against the OOM killer.  If you don't, then it's very likely that your kernel will kill ZFS-FUSE as soon as things get tight -- and this is something you definitely do not want.
    The listing below contains code to do just that.
  4. Remove limits.  If you don't remove the limits, ZFS-FUSE will either hang and spin, or consume an inordinate amount of memory (close to two gigabytes).
    ulimit -v unlimited
    ulimit -c 512000
    ulimit -l unlimited
    ulimit -s unlimited

For your convenience, I am attaching the script /sbin/zfsctl which I invoke (with the start argument) in /etc/rc.sysinit after mounting other filesystems, and invoke it (with the stop argument) before killall is executed in the S01killall initscript:  It works in Fedora and Ubuntu.

#! /bin/sh

PIDFILE=/.zfs-fuse.pid
LOCKFILE=/var/lock/zfs/zfs_lock

export PATH=/sbin:/bin
unset LANG
ulimit -v unlimited
ulimit -c 512000
ulimit -l unlimited
ulimit -s unlimited

log_action_begin_msg() {
	true # echo $*
}

log_action_end_msg() {
	true # echo $*
}

do_start() {
	test -x /sbin/zfs-fuse || exit 0
	PID=`cat "$PIDFILE" 2> /dev/null`
	if [ "$PID" != "" ]
	then
		if kill -0 $PID 2> /dev/null
		then
			echo "ZFS-FUSE is already running"
			exit 3
		else
			# pid file is stale, we clean up shit
			log_action_begin_msg "Cleaning up stale ZFS-FUSE PID files"
			rm -f "$PIDFILE"
			# /var/run/sendsigs.omit.d/zfs-fuse
			log_action_end_msg 0
		fi
	fi

	log_action_begin_msg "Starting ZFS-FUSE process"
	zfs-fuse -p "$PIDFILE"
	ES_TO_REPORT=$?
	if [ 0 = "$ES_TO_REPORT" ]
	then
		true
	else
		log_action_end_msg 1 "code $ES_TO_REPORT"
		exit 3
	fi

	for a in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
	do
		PID=`cat "$PIDFILE" 2> /dev/null`
		 [ "$PID" != "" ] && break
		sleep 1
	done

	if [ "$PID" = "" ]
	then
		log_action_end_msg 1 "ZFS-FUSE did not start or create $PIDFILE"
		exit 3
	else
		log_action_end_msg 0
	fi

	log_action_begin_msg "Immunizing ZFS-FUSE against OOM kills and sendsigs signals"
	# mkdir -p /var/run/sendsigs.omit.d
	# cp "$PIDFILE" /var/run/sendsigs.omit.d/zfs-fuse
	echo -17 > "/proc/$PID/oom_adj"
	ES_TO_REPORT=$?
	if [ 0 = "$ES_TO_REPORT" ]
	then
		log_action_end_msg 0
	else
		log_action_end_msg 1 "code $ES_TO_REPORT"
		exit 3
	fi
	
	log_action_begin_msg "Mounting ZFS filesystems"
	
	sleep 1
	rm -f /var/lib/random-seed
	zfs mount -a
	ES_TO_REPORT=$?
	if [ 0 = "$ES_TO_REPORT" ]
	then
		log_action_end_msg 0
	else
		log_action_end_msg 1 "code $ES_TO_REPORT"
		#echo "Dropping into a shell for debugging.  Post_mountall pending."
		#bash
		#post_mountall
		exit 3
	fi

	if [ -x /nonexistent -a -x /usr/bin/renice ] ; then # DISABLED
		log_action_begin_msg "Increasing ZFS-FUSE priority"
		/usr/bin/renice -15 -g $PID > /dev/null
		ES_TO_REPORT=$?
		if [ 0 = "$ES_TO_REPORT" ]
		then
			log_action_end_msg 0
		else
			log_action_end_msg 1 "code $ES_TO_REPORT"
			exit 3
		fi
		true
	fi
	
}

do_stop () {
	test -x /sbin/zfs-fuse || exit 0
	PID=`cat "$PIDFILE" 2> /dev/null`
	if [ "$PID" = "" ] ; then
		# no pid file, we exit
		exit 0
	elif kill -0 $PID 2> /dev/null; then
		# pid file and killable, we continue
		true
	else
		# pid file is stale, we clean up shit
		log_action_begin_msg "Cleaning up stale ZFS-FUSE PID files"
		rm -f "$PIDFILE"
		# /var/run/sendsigs.omit.d/zfs-fuse 
		log_action_end_msg 0
		exit 0
	fi

	log_action_begin_msg "Syncing disks"
	sync
	log_action_end_msg 0

	log_action_begin_msg "Unmounting ZFS filesystems"
	zfs unmount -a
	ES_TO_REPORT=$?
	if [ 0 = "$ES_TO_REPORT" ]
	then
		log_action_end_msg 0
	else
		log_action_end_msg 1 "code $ES_TO_REPORT"
		exit 3
	fi
	
	log_action_begin_msg "Terminating ZFS-FUSE process gracefully"
	kill -TERM $PID

	for a in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
	do
		kill -0 $PID 2> /dev/null
		[ "$?" != "0" ] && break
		sleep 1
	done

	if kill -0 $PID 2> /dev/null
	then
		log_action_end_msg 1 "ZFS-FUSE refused to die after 15 seconds"
		exit 3
	else
		rm -f "$PIDFILE"
		# /var/run/sendsigs.omit.d/zfs-fuse 
		log_action_end_msg 0
	fi

	log_action_begin_msg "Syncing disks again"
	sync
	log_action_end_msg 0
}

case "$1" in
  start)
	do_start
	;;
  stop)
	do_stop
	;;
  status)
	PID=`cat "$PIDFILE" 2> /dev/null`
	if [ "$PID" = "" ] ; then
		echo "ZFS-FUSE is not running"
		exit 3
	else
		if kill -0 $PID
		then
			echo "ZFS-FUSE is running, pid $PID"
			zpool status
			exit 0
		else
			echo "ZFS-FUSE died, PID files stale"
			exit 3
		fi
	fi
	;;
  restart|reload|force-reload)
	echo "Error: argument '$1' not supported" >&2
	exit 3
	;;
  *)
	echo "Usage: $0 start|stop|status" >&2
	exit 3
	;;
esac

:
Document Actions

Brilliant stuff

Avatar Posted by Seth Heeren at Feb 04, 2009 11:14 PM
Any reason why the sendsigs.omit.d approach is commented out?

I'd be happy to know which method works most reliably/portably.

Because

Avatar Posted by Rudd-O at Feb 04, 2009 11:15 PM
the sendsigs protocol is exclusive to Ubuntu. And it also requires special killall5.c code that is in the Ubuntu bug tracker to work correctly (the distributed killall5 is broken, I reported the bug and sent the patch, look in Launchpad).

killall5.c bug

Avatar Posted by David Abrahams at Feb 13, 2009 04:15 PM
It's this one, right?
https://bugs.launchpad.net/[…]/151580
As of this posting, still "In progress"

Yeah

Avatar Posted by Rudd-O at Feb 13, 2009 04:16 PM
EXACTLY.

rc.sysinit on ubuntu?

Avatar Posted by David Abrahams at Feb 12, 2009 09:21 PM
I don't seem to have that file on my intrepid systems. Are you sure this shouldn't be installed with update-rc.d or something on that distro?

rc.sysinit

Avatar Posted by Rudd-O at Feb 12, 2009 09:22 PM
Well, maybe it's not there in Ubuntu Intrepid Ibex. You need to study the Upstart sysvinit replacement to find a suitable place to run it.

Add comment

You can add a comment by filling out the form below. Plain text formatting.

Info
You are not logged in. You may optionally enter your username and password below. If you don't enter anything, this comment will be posted as 'Anonymous User'.
(Required)
(Required)
(Required)
Enter the word