The netrepaird script

First off, here’s the netrepaird script. It’s a simple script that attempts to ping network addresses (that you can define yourself), and if all pings fail, it’ll attempt progressively agressive measures to repair your network connection. It’s really low-tech, but it works.

There’s an updated, more featureful script here.

#!/bin/bash

# edit these variables to suit your needs HOSTS="rudd-o.com gplhost.com mozilla.org" # define hosts to ping here INTERFACE="eth1" # define the name of your network interface DRIVER="cdc_ether" # define the driver (kernel module) of your network interface

SUDO="sudo" # define to use sudo

netconnected() {

for host in $HOSTS; do NETCONNECTED=no $SUDO ping -c 4 $host > /dev/null [ "$?" == "0" ] && { NETCONNECTED=yes ; return ; } done

}

ifdownifup() {

$SUDO /sbin/ifdown $INTERFACE $SUDO /sbin/ifup $INTERFACE

}

reloaddriver() {

$SUDO /sbin/ifdown $INTERFACE $SUDO /sbin/modprobe -r $DRIVER $SUDO /sbin/modprobe $DRIVER $SUDO /sbin/ifup $INTERFACE

}

netconnected [ "$NETCONNECTED" == "yes" ] && { exit ; } # network is OK

echo "Network problem, toggling network interface $INTERFACE" ifdownifup ; netconnected [ "$NETCONNECTED" == "yes" ] && { echo "Network OK after interface toggle" ; exit ; }

echo "Network problem, reloading driver" reloaddriver ; netconnected [ "$NETCONNECTED" == "yes" ] && { echo "Network OK after driver reload" ; exit ; }

echo "Network is definitely down" logger -t netrepaird -p local0.warn "Network is down, could not repair connection" exit 1

Installing this script

Put this in the file named /usr/local/bin/netrepaird and make it executable (chmod +x /usr/local/bin/netrepaird).

Making it run every minute

That’s right, it’d make no sense to just chuck the script on your hard disk. For this, we’re going to make it run periodically, every minute. Edit your administrator’s crontab file:

• Ubuntu users: sudo crontab -e
• Fedora users: su -c 'crontab -e'

and place this line on the crontab file:

* * * * * /usr/local/bin/netrepaird

Small gotcha: make sure you put a carriage return at the end of the file, otherwise cron doesn’t execute the command.

That’s it

That’s it. Now, every minute, the script will run and check for an available Internet connection. If the connection flakes out for whatever reason, it’ll attempt to take it down and up, and if that fails, it’ll attempt to remove and reload the device driver and restart the connection.

And if those measures fail, your root user will get an e-mail with the warning, and a message will be duly noted in the system log file.

Packagers?

If any of my readers is interested in packaging this for a distribution (which would entail separating the configuration from the script), please contact me to get help.

Happy hacking!

/usr/sbin/pppd pty "ssh myinternetserver.com -t -e none -o 'Batchmode yes' \
/usr/sbin/pppd" 192.168.16.1:192.168.16.254 local nodetach silent \

This is all you need to create a secure tunnel between your computer and a computer that has the SSH running (in the example, myinternetserver.com) on the Internet.

Once you run this command, both your local and your remote computers will have new PPP network interfaces:

• Local: interface ppp0, IP address 192.168.16.1
• Remote: interface ppp0, IP address 192.168.16.254

But a few quick and easy preparations are needed in order to run and take advantage of this kind of tunnel.

The SSH server installed on the server, pppd on both machines

This should be self-explanatory. Use your Linux distribution’s package management tools to install them — should take less than 1 minute.

The right pppd configuration

You use the /etc/ppp/options file to configure it. Most options, once set in the configuration file, cannot be overriden in the command line.

On the server’s /etc/ppp/options file

lock
noauth
ipcp-accept-local
ipcp-accept-remote
noproxyarp

On the client’s /etc/ppp/options file

lock
noauth
noproxyarp

The right security configuration

[root@mycompanyserver.com #] /usr/sbin/groupadd -r ppp
...(add your SSH login user name to the ppp group in /etc/group)...
chown root.ppp /usr/sbin/pppd
chmod 750 /usr/sbin/pppd
chmod +s /usr/sbin/pppd

Now only members of the ppp group (namely, you) can join the fun.

SSH public key authentication enabled

• Put PubkeyAuthentication yes in the server’s /etc/ssh/sshd_options.
• Generate an SSH key pair in your computer.
• Upload the public key of the generated pair to your server’s ~/.ssh/authorized_keys2 file. Don’t forget to chmod 600 that file, or else SSH won’t even look at it.

That’s it!

Now run the following command on your computer:

/usr/sbin/pppd pty "ssh myinternetserver.com -t -e none -o 'Batchmode yes' \
/usr/sbin/pppd" 192.168.16.1:192.168.16.254 local nodetach silent \

What this command basically does is start pppd locally, tell it to connect via SSH in batch mode (using the public key, so it doesn’t ask for your password), start pppd on the remote server, and use the SSH connection as their conversation path.

Wait a few seconds. pppd will utter a few things:

Using interface ppp0
Connect: ppp0 <--> /dev/pts/7

You will have to confirm that the PPP interface is up:

[youruser@yourcomputer $] /sbin/ifconfig ppp0
ppp0      Link encap:Point-to-Point Protocol
          inet addr:192.168.16.1  P-t-P:192.168.16.254  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:5 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3
          RX bytes:75 (75.0 b)  TX bytes:69 (69.0 b)

Now you can ping 192.168.16.254. If pinging fails, check the firewall rules on both the server and your computer.

To interrupt the tunnel, hit Ctrl+C on the terminal that holds the running pppd command.

What next?

Bringing the tunnel up is actually one part of the deal. If you’re like me, perhaps you’ll want to actually carry all of your network traffic across the tunnel.

Set up the remote server to route traffic for you

This is easy. On the server, either:

• set firewall rules that will do NAT between the PPP interface and the Internet; or
• replace noproxyarp with proxyarp in the (server’s) /etc/ppp/options file; then set aside an extra IP address in the (server’s) local network; then use it as the remote endpoint in the pppd command. If your server has IP address 2.3.4.5, then provision 2.3.4.6 and use it instead of 192.168.16.254.

Set the default route to be the tunnel’s remote endpoint

If you want to route all of your network traffic through your newly created PPP link, you need to do two steps in this order:

1. Manually add a route to your server, so the SSH traffic knows the physical route it should follow. If the IP address of your current default route is 4.5.6.7, then run route add mycompanyserver.com gw 4.5.6.7
2. Replace the default route. Run route add default gw 192.168.16.254 dev ppp0

Before you suggest the pppd’s defaultroute option: no, it won’t work, because it’ll disrupt the only communication route between your computer and the remote SSH server.

Perhaps you’d like to add manual, shorter routes to your DNS servers — otherwise Web browsing may get a little slow. You know, due to the latency of the tunnel and stuff…

After doing this:

• your computer will send all (SSH) traffic to mycompanyserver.com through your regular default route, so the tunnel keeps working fine;
• the rest of the traffic will go through the PPP interface (notice we named the remote IP address in the default route command?)

Combine this with the proxyarp option on the server, and you have a recipe to make your computer appear as a completely different computer on the Internet. Perfect for anonymizing your real computer!

More Linux questions?

Have a Linux question? Why don’t you drop by our Month of Linux Answers and leave your question as a comment? I’ll get to it right away :-).

Why Web browsing slows down when downloading files

Believe it or not, this is common. If you’re like me, you have a fast fiber, DSL or cable modem hookup to the Internet, directly hooked to your computer. You’re probably using file sharing applications all the time, as well. This causes terrible Web browsing — and you can easily figure that out, because as soon as you stop any file sharing applications, Web browsing speed returns back to normal.

The slowdown is caused by the queues at your ISP (or your modem). The ISP limits your Internet speed either at the modem, or at the router assigned to your modem. Since information can’t come in as fast as possible, long queues of pending information build up in your router/modem.

The effect? After you click on a link in your Web browser, quite a lot of information (usually, your download) needs to be dispatched before you get to see the new Web page.

The solution: rate-limiting (odd, isn’t it?)

Counterintuitive as it may sound, the solution to the queues is to limit the rate at which information reaches your computer, directly on your computer. By using a clever combination of bandwidth limiting and priorization of outbound packages, you can have almost-normal Web browsing speeds, combined with fast downloads.

In other words:

• Anything your computer sends to the Internet will be given priorities, and important information will jump the queue. Sends of acknowledgements and interactive traffic (remote desktops, shells) will get priority.
• If any computer attempts to send data to your computer too quickly, your computer will tell it to slow down. This will avoid the buildup in the queue at your ISP.

Basically, you’ll be executing a tradeoff: decreasing latency at the expense of a bit (5% to 10%) of bandwidth, but the cost is certainly worth it.

Now, you may balk because this technique requires you to actually scale back (a bit) your connection speed. Do not — if you follow through, your effective download speed will go up, because TCP acknowledgements arrive to their destination faster. If you’re using BitTorrent (which requires uploading), you’ll see a stable, maxed-out, download speed, instead of seeing dips in download speed when uploading.

As usual, the solution is a clever script that does everything for you, instead of you having to do it manually.

The script that solves the speed problem

Further below, you’ll find instructions on how to use it.

#!/bin/bash

Wonder Shaper

please read the README before filling out these values

#

Set the following values to somewhat less than your actual download

and uplink speed. In kilobits. Also set the device that is to be shaped.

DOWNLINK=176 UPLINK=120 DEV=eth1

low priority source netmasks

NOPRIOHOSTSRC=

low priority destination netmasks

NOPRIOHOSTDST=

low priority source ports

NOPRIOPORTSRC="6881 6882 13810 6346 4800"

low priority destination ports

NOPRIOPORTDST=

high priority source netmasks

HIPRIOHOSTSRC=

high priority destination netmasks

HIPRIOHOSTDST=

high priority source ports

HIPRIOPORTSRC="5900 5901"

high priority destination ports

HIPRIOPORTDST=

if [ "$1" = "status" ] then echo "Queueing disciplines:" tc -s qdisc ls dev $DEV

    rootclassoutput=`tc -s class ls dev $DEV | grep -A 5 "htb 1:1 "`
    allclassesoutput=`tc -s class ls dev $DEV`

    echo ""
    echo "Root class:"
    echo "$rootclassoutput"
    echo ""
    echo "Child classes:"
    echo "$allclassesoutput" | cut -d "

” -f 6,7,8,9,10,11 –complement exit fi

clean existing down- and uplink qdiscs, hide errors

tc qdisc del dev $DEV root 2> /dev/null > /dev/null tc qdisc del dev $DEV ingress 2> /dev/null > /dev/null

if [ "$1" = "stop" ] then exit fi

uplink

we’ll have four classes:

10: acknowledgements and traffic marked as interactive (Minimize-Delay)

20: high-priority outbound traffic

30: regular traffic

40: low-priority outbound traffic and traffic marked as bulk (Maximize-Throughput)

install root HTB, point default traffic to 1:30:

tc qdisc add dev $DEV root handle 1: htb default 30

shape everything at $UPLINK speed - prevents huge outbound queues:

tc class add dev $DEV parent 1: classid 1:1 htb rate ${UPLINK}kbit burst 5k

tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${UPLINK}kbit ceil ${UPLINK}kbit burst 5k prio 1 tc class add dev $DEV parent 1:1 classid 1:20 htb rate ${UPLINK}kbit ceil ${UPLINK}kbit burst 5k prio 2 tc class add dev $DEV parent 1:1 classid 1:30 htb rate $[8$UPLINK/10]kbit ceil ${UPLINK}kbit burst 5k prio 3 tc class add dev $DEV parent 1:1 classid 1:40 htb rate $[6$UPLINK/10]kbit ceil ${UPLINK}kbit burst 5k prio 4

all get Stochastic Fairness:

tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10 tc qdisc add dev $DEV parent 1:40 handle 40: sfq perturb 10

TOS Minimum Delay (ssh, NOT scp) in 1:10:

tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32 \ match ip tos 0×10 0xff \ flowid 1:10

To speed up downloads while an upload is going on, put ACK packets in 1:10:

tc filter add dev $DEV parent 1:0 protocol ip prio 11 u32 \ match ip protocol 6 0xff \ match u8 0×05 0×0f at 0 \ match u16 0×0000 0xffc0 at 2 \ match u8 0×10 0xff at 33 \ flowid 1:10

High-priority traffic in 1:20:

for a in $HIPRIOPORTDST ; do tc filter add dev $DEV parent 1: protocol ip prio 12 u32 match ip dport $a 0xffff flowid 1:20 done for a in $HIPRIOPORTSRC ; do tc filter add dev $DEV parent 1: protocol ip prio 12 u32 match ip sport $a 0xffff flowid 1:20 done for a in $HIPRIOHOSTSRC ; do tc filter add dev $DEV parent 1: protocol ip prio 12 u32 match ip src $a flowid 1:20 done for a in $HIPRIOHOSTDST ; do tc filter add dev $DEV parent 1: protocol ip prio 12 u32 match ip dst $a flowid 1:20 done

Low-priority traffic in 1:40:

for a in $NOPRIOPORTDST ; do tc filter add dev $DEV parent 1: protocol ip prio 13 u32 match ip dport $a 0xffff flowid 1:40 done for a in $NOPRIOPORTSRC ; do tc filter add dev $DEV parent 1: protocol ip prio 13 u32 match ip sport $a 0xffff flowid 1:40 done for a in $NOPRIOHOSTSRC ; do tc filter add dev $DEV parent 1: protocol ip prio 13 u32 match ip src $a flowid 1:40 done for a in $NOPRIOHOSTDST ; do tc filter add dev $DEV parent 1: protocol ip prio 13 u32 match ip dst $a flowid 1:40 done

Maximize throughput (scp, BitTorrent, uploads) in 1:40:

tc filter add dev $DEV parent 1:0 protocol ip prio 14 u32 \ match ip tos 0×08 0xff \ flowid 1:40

rest is ‘non-interactive’ ie ‘bulk’ and ends up in 1:30

tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \ match ip dst 0.0.0.0/0 \ flowid 1:30

#### downlink

slow downloads down to somewhat less than the real speed to prevent

queuing at our ISP. Tune to see how high you can set it.

ISPs tend to have huge queues to make sure big downloads are fast

#

attach ingress policer:

tc qdisc add dev $DEV handle ffff: ingress

filter everything to it (0.0.0.0/0), drop everything that’s

coming in too fast:

tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip src \ 0.0.0.0/0 police rate ${DOWNLINK}kbit burst 5k drop flowid :1

This is a heavily tuned version of the Wonder Shaper script that roams the Linux advanced routing and traffic control Web site.

Keep reading to find out how to install and use this script on your computer.

My answer: yes, categorically.

The open source advantage: why you should be using open source applications

Well, there are at least two reasons:

1. Open source applications tend to be higher-quality. I don’t want to generalize, because I’ve test-driven tons of immature open source applications, but a great portion of them are very high-quality, and tend to improve with time, since everyone can “pitch in” and help develop applications further.
2. They’re (usually) free. Why pay money to “the man”, when you can save money and do better?
3. They respect standards better. You can easily move to Linux or other operating systems, because they respect standards and save their information in easy-to-use, easy-to-backup, easy-to-move file formats.

So, which applications?

Open source applications for Microsoft Windows number in the thousands. The most widely used have gotten great amounts of press coverage, are mature, stable and well-known:

• OpenOffice.org: this Microsoft Office killer has nearly everything to be the next king in office-suite land.
• Mozilla Firefox: unless you’ve lived under a rock for the past 5 years, it’s hard to believe you haven’t heard about it. Plainly said, it’s the best Web browser, ever.
• The VideoLAN client: it’s a straightforward, no-frills, fast media player. The amazing thing about VLC is that it supports lots and lots of video and audio formats — you can even play Flash videos with it(for example, downloaded from YouTube with the VideoDownloader Firefox plugin).
• Azureus: downloading torrents in your computer? Azureus is the best there is for Windows.

Chances are, you’re already using open source applications in your computer. Note I said applications — some portions of Microsoft Windows are direct descendants of other open source operating systems (such as FreeBSD). But the short list doesn’t stop there.

Where do I get them?

Three distinct projects aim to bring lots of originally-Linux open source applications to Windows:

1. Open Source Windows. Billed as This is the best Windows software that we know of. No adware, no spyware, just good software., it’s absolutely on the spot.
2. TheOpenCD. Instead of having to download each application, you can simply download the CD and start test driving them. Obviously, you can copy the CD as many times as you like, and give it to acquaintances, friends and family.
3. PortableApps. If you have an USB thumb drive, you’ll find immense value in this project; if you’re on the road, you no longer need to install and configure each application on the computer you’re about to use — just install them once in your thumb druve, then plug it on any Windows computer.

And the march to Windows continues!

KDE on Windows! Replace your Windows Explorer with a much more powerful desktop environment!

Actually, what gets me even more excited is the fact that Amarok, the best music player/manager ever to see the light, is going to be available on Windows as well. Amarok cleanly blows iTunes, Windows Media Player and Winamp out of the water.

I’ve open sourced my applications… what next?

Run them for a couple of days, maybe two weeks, and avoid running the applications they replaced (Internet Explorer, Outlook Express, and so forth).

Get comfortable with them. Most likely, you’ll be using them in Linux too. Take this opportunity to learn, effortlessly, more about computing. No one ever complained when they saw a résumé and noticed extra computing experience, right?

Finally, install Linux. I recommend KUbuntu, because it’s more similar to Windows than regular Ubuntu. Download the live CD, burn it to a CD, then boot from the CD. Run it for a while, see if you like it, install it — once you’ve installed it to your computer, you can use Adept (included in KUbuntu) to install your beloved open source applications.

Did I mention installing applications on KUbuntu is much easier than on Windows?

More Linux questions?

Have a Linux question? Why don’t you drop by our Month of Linux Answers and leave your question as a comment? I’ll get to it right away :-).