How to Upload Files from a Web Page to a Python Script

My task was to select files for upload using a file input element on a web page, and then upload them to a Python script on the server that would write them to disc. It was fairly straightforward, but I got stuck trying to get it to work with binary files. The trick is to explicitly set stdin and stdout to work with binary data.

The form on my web page stored the file data in a field called 'uploaded[]'. I was using HTML5, so the file input was able to upload multiple files at once. In this case the value the Python script obtains from the FieldStorage object is a list of file objects. However, if you only upload a single file, the value is a single file object. I make a simple check on the data to make sure I'm dealing with a list.

#!python

""" 
    Gets passed file contents as POST data from an HTML page. 
    Writes them to a file on disc. 
"""

import cgi

try: # Windows needs stdio set for binary mode.
    import msvcrt
    msvcrt.setmode (0, os.O_BINARY) # stdin  = 0
    msvcrt.setmode (1, os.O_BINARY) # stdout = 1
except ImportError:
    pass

form = cgi.FieldStorage()
fileItems = form['uploaded[]']

# You only get a list back if you're uploading multiple files. Make sure we're always working with a list.
if type(fileItems) != type([]):
    fileItems = [fileItems]

for fileItem in fileItems:
    if fileItem.file:
        f = open(fileItem.filename, 'wb')
        while True:
            chunk = fileItem.file.read(100000)
            if not chunk: 
                break
            f.write(chunk)
            f.close()

# Output header.
print 'Content-type: text/html'
print

Genealogy is Magic

It truly is: it can make an entire evening disappear, with maybe just a birth certificate to show for all the time gone.

I never got genealogy. I couldn't see the attraction of rummaging through old records or wandering about in graveyards looking for distant ancestors. But I'm adopted, and a month ago I finally decided to try to trace my birth mother. Armed with a name and a rough address from 53 years ago, I signed up to ancestry.co.uk and started searching. At first I thought its search facility was appalling: so bad I had to be doing something wrong. I was. I still think it leaves a lot to be desired in ease of use, but with exploration of its features it turns out to be extremely powerful. I could soon see a record of my mother entering the country at Southampton (gave me a date of birth), and shortly afterwards I found her full address in New Zealand in 1957, as well as the names of her parents.

The New Zealand electoral registers are all searchable online, but only up to 1981. Also, the register is only compiled every three years. Still, I could track the family's movements to a fair extent, and found the name of another individual I presumed (correctly) was my uncle.

Then I found that the New Zealand government had an online search website that lets you look up births, deaths and marriages, providing they're at least a few decades in the past, or, for deaths, the deceased was born at least 80 years ago. For a small payment they will email you a PDF of the relevant document.

At this point I sort of lost the original plot, and began tracing my family tree. Matching up disparate records had turned into a hugely engaging intellectual puzzle, and best of all, one you can do sat at a computer. You can trust nothing, because filling in forms is a boring activity and mistakes get made. An 'o' was transcribed as an 'a', a person is born with one set of first names but dies with another, possibly somebody lies about their age. Evidence has to be weighed up, possibilities have to be sorted through. Lovely.

I finally 'got' genealogy.

As I predicted in an earlier post (The End of Genealogy), I'd gone back three generations when I bumped into somebody else's family tree: a truly massive affair with a cast of dozens, possibly hundreds, of people (it's hard to tell). If a married couple in that tree were also two of my great grandparents, then I could go back another three generations into 18th century Ireland. Exciting stuff, but as I explored this huge tree that someone (or surely an army of genealogists) had patiently built up, the scope of the job I'd begun came home to me: everybody has a big, big number of ancestors.

I went back to trying to locate my living relatives.

Revisiting the search websites armed with more experience of how to use them properly, I found a death certificate for my grandmother. From that I found a death notice on a site dedicated to finding them, which led me to a report on a local newspaper site, which gave me the names of an aunt and uncle. The aunt was on Facebook, so I finally bit the bullet and created a Facebook account. In another post I might talk about how much I dislike Facebook's user experience, so I won't bother now complaining about how it's constantly presenting me with names of people I've never heard of asking if I want to friend them, even though I've already told it several times already that I don't, or how sending a message to a person you're not friends with results in them not seeing it, or how their friend finder was unable to find "Charles Anderson York" even though it found loads of other Charles Andersons. No, I won't talk about that right now. Let's just say that, after a week of waiting for a reply to a message sent to the 'Other' inbox, I paid 35p to send the same message directly to my aunt's main inbox and I finally made contact with a birth relative other than my son.

My birth mother is no longer with us, but at least now I know, and I've discovered relatives that before I could only speculate about. I've also found out where the 3.8% Jewish ancestry that 23andMe identified comes from. (The 2.9% Neanderthal inheritance remains unaccounted for.)

I'll finish by saying that I am truly amazed at how much of this type of information is now available via the internet. And it will only get better as the genealogy sites get smarter and more knowledgeable. Since finding my birth relatives I've tried doing the same for my adopted parents. Both of them are already on other people's trees; all I have to do is cross some T's and dot some I's. And even if you're not into genealogy, genetic testing sites like 23andMe will connect you with your relatives as an optional extra. Soon we will all be related.

In Memory of Jack Vance

I found out this week that Jack Vance died last month; how I've loved his books these least forty years or so.

Although his main characters could sometimes seem a bit underdrawn, the minor characters, his landscapes and his settings were always wonderfully inventive. And his style! If I had any pretensions to writing fiction, I would quietly shelve them after reading his.

Looking at his bibliography I see that there are still many, many works that I haven't read. Of the ones I have, I enjoyed the fantasy most, although individual SF novels such as Emphyrio or Night Lamp were also excellent. The Dying Earth books will always be my favourites, and Cugel my most loved Vance character, despicable though he often was. Vance wonderfully represents a sense of world-weariness as the inhabitants of the Earth wait for the sun to finally die out. But in the meantime there are still spectacles to see, wines to sample, manses to rob...

My Old Exam Papers

Last week while hunting for something up in our loft, I came across a collection of exam papers. Here were all my old GCE and A level papers, plus most of my University exams. It makes interesting reading—decades on.

For a start, I still know quite a lot of the O level stuff; I'm pretty sure I could still get A in Maths and German, though not so sure how good my grades would be in the other subjects. The A levels are another story. My Maths papers are full of names and terms which ring bells, but unfortunately distant ones. "De Moivre's Theorem". Ah, I remember that! No, turns out I only remember the name. Looking it up brought some recognition, but not a lot. The Physics A level was better, which was odd given that I went on to University to do more Maths, not Physics.

As for my Uni papers? Well, to be honest I couldn't even remembering knowing most of it. I had put a star next to the questions I'd answered during the exams, but I wouldn't have been able to tell otherwise: they looked equally baffling. The Computer Science papers were considerably more understandable, which is probably just as well seeing as how I went on to a career of programming. I can still do all the questions that I can actually understand, but in some the terminology must have changed or gone out of fashion.

So what can I conclude? That most of my education after the age of 16 was a waste of time? Not very happy with that suggestion.

I'd like to think that the heart of the matter is a feature of Mathematics. University standard Maths is so abstract that, if you don't use it all the time, you quickly forget it. Since I graduated I've had precious few occasions to use what I learned, and so it's gone. I can only be grateful that back in those days you didn't leave University with five figures of debt. Paying off a loan that size for years when you'd already forgotten most of what you'd learned: that must sting.