Rudd-O.com

If you’re one of our regular readers, you’ll remember reading the article and script I wrote on automating BitTorrent downloads with TorrentFlux and rsync. The script has come to be quite handy to me, but the process of writing it is much more fun and appealing. Here’s how I did it.

What you’re about to read is a short, true story that details how I went from idea to solution, using the “UNIX way” (divide a problem in little pieces, then jump each hurdle with the help of a domain-specific solution, and finally combine each response).

The story is — honest! — more about the underlying thought process than the technology I used — in fact, while I do know Python and I’ve grown white hairs using Linux, most of the stuff you’ll see here I taught myself in the course of this challenge.

TorrentFlux: what it is, how it works

TorrentFlux is a very interesting Web application you can install on your Web server. Once installed, it lets you in to a nice interface like any other BitTorrent program where you get to add .torrent files so the torrent gets downloaded directly in your Web server. Here it is:

Advantages of using TorrentFlux

The reasons to use such a program instead of a normal desktop BitTorrent client are varied:

• you would like to have a particular, big file (think Linux distro) right away on the server instead of uploading it through your ISP’s connection
• your ISP throttles or blocks BitTorrent (think Comcast)
• you would like to avoid downloading through 50 TCP connections at home, and just use one (think speedier Web browsing)
• you want to take advantage of the (usually) higher bandwidths available in your Web server during the day, and you’d like to download the file once it’s done, from beginning to end (think videos)
• you would like to queue up a download while away from home (think vacations)

Disadvantages

The disadvantages, compared to a regular BitTorrent client, are that:

• if you want the download in your PC, you need to queue it up separately in a download manager on your PC through FTP,
• you need to check TorrentFlux periodically to see if the torrent is done downloading, because you can’t download a torrent to your PC until the torrent is done,
• you can’t queue up downloads from TorrentFlux to your home PC if you’re away from your computer, can you?
• disk space might be limited in your Web server.

What we’ll do in this article is destroy these disadvantages.

The problem in a nutshell

From the aforementioned disadvantages we can clearly see what the problem is. Being the lazy bastards we are, we want the downloads in our home computer ASAP, without manual intervention. Our ultimate goal is to:

1. queue up a BitTorrent download in TorrentFlux, and
2. as soon as it’s finished, have our home PC automatically “check it out from the counter”, and then
3. erase the file from the TorrentFlux server.

Let’s go do it.

How I solved it

Remoting my server

How am I going to talk to my server? This question crossed my mind many times. In the end, I chose SSH, because it’s extremely easy to automate and have it work securely without constant interruptions — remember, we don’t want to be interrupted with password prompts or anything like it.

I wrote the code that more or less provided enough help for it to work. As programming languages go, Python is by far the best for the task at hand, so I used it. OK, I’ll admit it, I’m a Python junkie:

#!/usr/bin/env python

from subprocess import Popen,PIPE,STDOUT,call import fcntl import re import os import sys import signal

def getstdout(cmdline): p = Popen(cmdline,stdout=PIPE) output = p.communicate()[0] if p.returncode != 0: raise Exception, "Command %s return code %s"%(cmdline,p.returncode) return output def getstdoutstderr(cmdline,inp=None): # return stoud and stderr in a single string object p = Popen(cmdline,stdin=PIPE,stdout=PIPE,stderr=STDOUT) output = p.communicate(inp)[0] if p.returncode != 0: raise Exception, "Command %s return code %s"%(cmdline,p.returncode) return output def passthru(cmdline): return call(cmdline) # return status code, pass the outputs thru def getssh(cmd): return getstdout(["ssh","-o","BatchMode yes","-o","ForwardX11 no",torrentflux_server] + [cmd]) # return stdout of ssh. doesn't return stderr def sshpassthru(cmd): return call(["ssh","-o","BatchMode yes","-o","ForwardX11 no",torrentflux_server] + [cmd]) # return status code from a command executed using ssh

Interesting, isn’t it? Now I have two functions:

• one that gets me the text of a command executed by SSH, and
• another that discards the text but returns the return value of the executed command — which, if you know the UNIX convention, it’s zero for success, and anything else for errors.

Determining which torrents are done

The first hurdle I need to surpass: which torrents from the torrent list are already done? After a little poking around the Web, I discovered that with the help of a command called fluxcli that ships with TorrentFlux, I can get a list of active torrents in text form.

So that’s solved. All I needed is some code that parses the rows returned, selects only those rows that say Done or Seeding in the status column, and that should return the name of each torrent that’s ready to download.

The code in question is this:

def get_finished_torrents():
        stdout = getssh("%s transfers"%fluxcli)
        stdout = stdout.splitlines()[2:-5]
        stdout = [ re.match("^- (.+) - [0123456789.]+ MB - (Seeding|Done)",line) for line in stdout ]
        return [ match.group(1) for match in stdout if match ]

Stupidly simple, and it makes use of the SSH remoting functions (dare I say hacks?) I wrote before. And, may I say, only three lines, really. It returns a list with the names of the torrents.

Next up, getting the file names inside the torrents.

Pages: 1 2 3

This entry was posted on Friday, November 2nd, 2007 at 11:37 and is filed under Linux, Programming, Python, Tutorials. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.