drive a webcam with python

September 17th, 2006

I bought a USB webcam off of eBay quite some time ago, and I decided to connect it to my telescope with a little bit of hardware hackery. I’ll have to see about posting a writeup on how I did that at a later time. Anyway, when I installed my camera software, I quickly found how horrible the program was. It gave a tiny preview of what the camera saw, and had no way of capturing images or video without waaaay too many clicks of the mouse. That’s when I decided to write my own in Python.

The main libraries that I ended up using were VideoCapture, PIL, and pygame. As you’ll see, it’s not really difficult at all to make a useful webcam application.

Here’s the code:

from VideoCapture import Device
import ImageDraw, sys, pygame, time
from pygame.locals import *
from PIL import ImageEnhance

res = (640,480)
cam = Device()
screen = pygame.display.set_mode((640,480))
font = pygame.font.SysFont("Courier",11)

def disp(phrase,loc):
    s = font.render(phrase, True, (200,200,200))
    sh = font.render(phrase, True, (50,50,50))
    screen.blit(sh, (loc[0]+1,loc[1]+1))
    screen.blit(s, loc)

brightness = 1.0
contrast = 1.0
shots = 0

while 1:
    camshot = ImageEnhance.Brightness(cam.getImage()).enhance(brightness)
    camshot = ImageEnhance.Contrast(camshot).enhance(contrast)
    for event in pygame.event.get():
        if event.type == pygame.QUIT: sys.exit()
    keyinput = pygame.key.get_pressed()
    if keyinput[K_1]: brightness -= .1
    if keyinput[K_2]: brightness += .1
    if keyinput[K_3]: contrast -= .1
    if keyinput[K_4]: contrast += .1
    if keyinput[K_q]: cam.displayCapturePinProperties()
    if keyinput[K_w]: cam.displayCaptureFilterProperties()
    if keyinput[K_s]:
        filename = str(time.time()) + ".jpg"
        cam.saveSnapshot(filename, quality=80, timestamp=0)
        shots += 1
    camshot = pygame.image.frombuffer(camshot.tostring(), res, "RGB")
    screen.blit(camshot, (0,0))
    disp("S:" + str(shots), (10,4))
    disp("B:" + str(brightness), (10,16))
    disp("C:" + str(contrast), (10,28))

I decided to use pygame in order to build this because it can actually handle the fps that I need for video. I built something similar quite a while ago with wxPython, but it just wasn’t fast enough at image swapping to make it work well.

A couple of noteworthy points:

  • The function on line 15 is simply there to help automate displaying information on the screen. In my case, I display the number of screenshots I’ve saved to disk, and the current brightness and contrast levels. The function displays the text sent to it by writing it in dark gray, and then re-writing it in light gray over the top with a pixel offset. I found this helps a TON to make the text visible in different scenarios.
  • Line 26 is where I actually apply the contrast and brightness changes. Lines 30-33 set up the number keys 1-4 to adjust contrast/brightness on the fly. This is terribly useful, in my experience.
  • Lines 34 and 35 are setting up ‘q’ and ‘w’ to show the settings dialogs for the webcam… this is where you can set resolutions, exposure levels, etc.
  • On line 36, and those under the if statement, I save the current image and name it using the current time… this insures no overlapping image names.

If you’re trying to write a webcam app of your own, I hope this gets you pointed in the right direction. Let me know if you find this info useful!

get it : PhotoRec

September 7th, 2006

It’s funny how often I find myself deleting things. I mean, Shift-Delete is hard-wired in my brain, just because I delete constantly and hate it when my Recycle Bin sits there full. But the worst thing of all is when I go in to innocently delete a directory that I deem useless, and I remember days after the fact that there was something important in it. That’s why PhotoRec is such a life saver. Before you prejudge it by it’s name, this is a real file recovery program… it’s not just for photos. The list of recognized file types is pretty huge, in fact.

Basically, PhotoRec uses data carving techniques in order to find the files. Data carving is the process of extracting a collection of data from a larger data set… files are “carved” from the unallocated space on the drive. The nice thing is, it works independently of the filesystem headers, so it’s possible to recover files even if the partition info has been screwed up. The negative side of it all is that quite often data carving results in a lot of false positives. So tbh, you’ll end up with a directory of tons of files, and you’ll need to run through that directory in order to find the files that are actually good. But hey, since the purpose is to recover lost files, at least it sides on the ‘false positive’ side rather than the ‘false negative’, right?

Anyway, it’s terribly useful, and I’ve personally been able to recover many lost files with it. I hope your results are just as good!

install Pure-FTPd on Ubuntu

September 5th, 2006

I’m sure these instructions are similar for most distributions, but I thought I’d specifically target Ubuntu so that I don’t have to cover the little differences that each and every person might run into. The end result of this is to have a fast and secure FTP server running on Linux that, when given the proper credentials, will serve up a user’s home directory. You can imagine when this would be useful. I decided to go with Pure-FTPd for quite a few reasons, but mainly because of solid security and many configuration options. I’ve gotten myself into trouble before by choosing very simple FTPd servers and realizing later that they just weren’t able to do what I wanted them to do. Pure-FTPd is different that way.

So to get started, in Ubuntu just use apt to get Pure-FTPd installed:

apt-get install pure-ftpd

Now, set up a new user, and open up /etc/passwd for editing. Add /./ to their home directory like this:


This keeps them from seeing the filesystem.

Also, make sure to put in a shell at the end of that line… /bin/bash works fine. After all of that, add their username to /etc/ftpallow.

Now it’s time to give Pure-FTPd a basic configuration. Edit /etc/pam.d/pure-ftpd to look like this:

auth sufficient
auth required shadow use_first_pass
auth required item=user sense=deny file=/etc/ftpusers onerr=succeed
auth required item=user sense=allow file=/etc/ftpallow onerr=fail
auth required

Now just type pure-ftpd-control restart on the command line and you’re all finished!

chirp chirp chirp

September 5th, 2006

Have you ever been sitting there, happily chatting away with some friends, and someone makes a joke that’s well… it’s just SO unfunny that you have to point it out? A joke that is meant to be hilarious, but bombed so hard that everyone is just left looking at the ground or at each other, wondering what on earth just happened? It’s occurred quite a few times around me, and every time I can’t help but think of those same wonderful moments alluded to on the Simpsons, where you hear the wind blowing through the character’s hair, or hear a wolf howling, or (my personal favorite) hear a cricket chirping.

That’s why I set out to build a program that sits in your system tray, anxiously awaiting the moment when you click on it and it gets to chirp it’s little heart out. So without further rambling, I present Cricket 1.0. Pretty simple… just download it and run it. You’ll see a little icon in your tray that you can double-click at will.

Go ahead! Make fun of your coworker’s unfunny jokes! It’s quite possibly one of the most enjoyable things ever. ;)

Have fun, and if you find this program useful, please comment! Also, if you would like the source code, just ask. I’m not posting it right now just because I figured no one would really want it… but if you’d like it for reference or modification, I would be happy to provide it.

get it : ToDoList

September 5th, 2006

I just ran across a really great todo list program via Digg, and thought I’d mention it. It’s got a TON of functions that are all really really useful, plus it’s free and open-source. I’ve needed a program that would allow me to manage all of my projects and sub-projects at once, and this one fits the bill perfectly. w00t! :)

use macros in asterisk

September 4th, 2006

I was just watching an episode of Systm the other day with a friend… Episode 5, the one where they describe how to build a basic Asterisk server. First, I’ve gotta say, I love that Kevin Rose and the other guys over there have created Systm… it’s a very cool idea, and something that honestly just sounds like a blast to do.

Anyway, while watching it, I remembered back when I was learning asterisk that many would-be mentors simply showed how to create a static definition in extensions.conf for every extension. While that works for a small system, it quickly becomes needlessly complicated when you need to change the way things flow for every extension after that. Just imagine changing something as small as the voicemail context for 50 extensions. It just doesn’t make sense. That’s where macros come in. They make it possible to cover most of the extensions with one script. The issue, I’ve found, is that most example macros just aren’t useful enough. :)

For my users, I want to give them the ability to call their own number in order to go straight to voicemail. That, and the Polycom 501 actually defaults to calling itself when you hit the ‘messages’ button. So we need to create a macro that will perform the normal functions, but will also know when you’re calling yourself.

First, in the main context you’ll need a line like this:

exten => _10XX,1,Macro(sipphone,${EXTEN})

This assumes extensions between 1000 and 1099. Change it to suit your needs. The point of this line is that when someone dials a ‘1000’ extension, it will send the call to the ‘sipphone’ macro, sending the actual extension dialed along with it.

Then, somewhere else in the file, perhaps just above your main context, add:


exten => s,1,GotoIf($[“${ARG1}” = “${CALLERIDNUM}”]?5:2)
exten => s,2,Dial(SIP/${ARG1},20)
exten => s,3,Voicemail(su${ARG1})
exten => s,4,Hangup
exten => s,5,VoicemailMain(${CALLERIDNUM})
exten => s,6,Hangup

So when a call is sent to this macro, ${ARG} would equal the extension dialed, and ${CALLERIDNUM} would be the extension dialing. On the first line of the macro, we’re simply saying “if the caller id and dialed number are equal, send the call to line 5, or voicemail. Otherwise, we follow the normal flow of a macro and ring the extension for 20 seconds, and then send the call to voicemail. Also, notice that if you call yourself, we use the VoicemailMain command instead of the Voicemail command… that sends you in to listen rather than record.

making a radar loop with PHP

September 3rd, 2006

So a few days ago I was out with friends and a storm was on it’s way in. Naturally, I’m the one they ask whether it’s ‘about to hit us’, so I pull up the web browser on my Treo 650 and realize… um… there’s no quick way to pull up a radar loop. I mean, I used the great mobile site from the NWS, but it still didn’t give me the effect I wanted. And hey, I’m already filling a database with weather observations and radar images! So I decided to see what it would take to make a custom radar loop that I could get to at any given moment. The Treo can display animated gif files, so that seemed like the most efficient solution. The question is, how do we create an animated gif using PHP?

Well, I don’t like to reinvent the wheel, especially if someone else out there has made a wheel superior to one that I could make myself. :) Hence, I found this. From here, we just need to read in the images and feed them to the GifMerge class.

Header(‘Content-type: image/gif’);

$count = 1;
$q = mysql_query(‘select * from (select recorded, radar from weather order by recorded desc limit 10) as t order by recorded’);
while($r = mysql_fetch_array($q)){
$image_p = imagecreate(300, 300);
$image = imagecreatefromgif(“radar/”.substr($r[‘radar’], 35));
imagecopyresampled($image_p, $image, 0, 0, 210, 120, 300, 300, 300, 300);
imagegif($image_p, “radar/loop/frame”.$count.”.gif”);

$__i = array(“radar/loop/frame1.gif”, “radar/loop/frame2.gif”, “radar/loop/frame3.gif”, “radar/loop/frame4.gif”, “radar/loop/frame5.gif”, “radar/loop/frame6.gif”, “radar/loop/frame7.gif”, “radar/loop/frame8.gif”, “radar/loop/frame9.gif”, “radar/loop/frame10.gif”);
$__d = array(30, 30, 30, 30, 30, 30, 30, 30, 30, 300);
$gifmerge = new GifMerge($__i, 1, 1, 1, -10, $__d, 2, 0, 0, ‘C_FILE’);
echo $gifmerge -> getAnimation();

One issue I had was that the images I scrape are rather large, and when you merge 10 of them into an animated GIF, you end up with an image that a little bigger than most handheld devices like to deal with. Not to mention, we’re talking about a GPRS connection here… it would be easier to download 50kb than 500kb. So to get around that, I used the imagecopyresampled function to cut out the portion of the image most important to me. I then save the cropped image into a ‘loop’ directory as 10 separate ‘frame’ files. After that, it’s pretty straightforward to follow the GifMerge example to create the image.

So without further ado, here’s the result:

filling a database with historic weather data

September 2nd, 2006

I’m one of those people that just loves inclement weather. Maybe it’s because of where I grew up, but I’ve always adored a good thunderstorm. Whenever a storm is heading for my area, I check various sites online to get the latest radar and what-not, but sometimes you just aren’t near a computer (gasp!). Not only that, but most of radar loops you find only go back so far… maybe an hour or two. That’s why I decided to build my own solution, using data that the NOAA provides for free. Pretty much everything here is written in Python, the greatest language ever. :)

So first of all, I had to decide how to actually collect the data. There are a few methods, but one issue I faced is that I normally look at this stuff on my laptop. What if I have my laptop turned off? It’s just not a reliable platform to use for mining data. Happily, I have a dreamy webhost that gives me shell access. So there you go… the perfect spot to house the fruitage of my scripts. So what I’m going to do is build a script that runs regularly, contacts a weather site to scrape a radar image, and contacts the NOAA to get the latest weather data… temperature, barometer, etc. At that point, I want to insert this stuff into a database housed at the same host.

I’m going to take you through this as if you’re fairly new to Python, and the plan is to be really easy to understand. If it isn’t… well… you get what you pay for, right? :) Seriously though, if you end up with questions, just post them at the end of this article and I’ll answer them.

So I logged into my shell account at my webhost via SSH. I created a new text file called ‘radardl’ using the command vi radardl, and then made it executable with chmod +x radardl. As for the code that goes in that file…

import MySQLdb, time, os
from urllib import urlopen, urlretrieve

Nothing special here… just set it up to execute using your Python executable, and import the necessary modules. MySQLdb is used for working with the database, and urllib is used to get the web pages / images that we’ll need.

db = MySQLdb.connect(host=’dbhost’, user=’dbuser’, passwd=’dbpasswd’, db=’dbname’)
c = db.cursor()

This is the basic connection to your database. Obviously, you’ll need to set one up somewhere so that you’ll actually be able to do this. Replace the placeholders with your specific connection info. You’ll also need to create a new table to hold your data… I used something like, “create table weather (recorded timestamp primary key, temperature_string varchar(50), temp_f float, relative_humidity float, pressure_in float, heat_index_f float, windchill_f float, radar varchar(250))“. For simplicity, go ahead and use that command to create your table and you can modify it after we’re done.

j = urlopen(‘’)

Ok, this is where we download the current weather observations. Take a look at this page and select your state, then find the weather station closest to you. For me, it’s KTUL in Oklahoma. Just find your local page, and replace the url in the code. While you’re looking at it, take a look at all of the data that’s provided in there. It’s a wealth of observation data! I’m only capturing a couple of things with this code, but you can easily add what you want.

stats = {}
for l in j.readlines():
stats[l[l.find(“<")+1:l.find(">“)]] = l[l.find(“>”)+1:l.rfind(“<")] [/python] Now, you could use something fancy like the wonderful Elementtree library, but in my case my host doesn’t have it installed and I couldn’t do it myself. So instead I used the above code to give me a similar result. Basically, I search for the < and > characters in different ways in order to get the key name and the value. I then store those into the ‘stats’ array. This works great for the XML that the NOAA provides… but keep in mind that XML from other sources may not be so easy to parse.

rightnow = str(time.time())

for line in j.readlines():
if line.find(“img src=”) > -1:
img = line.strip()[11:-1]
urlretrieve(img, rightnow+”.gif”)

Ok… the thing is, everyone has their favorite weather radar, and I don’t want to advocate one over the other. I’d also prefer not to give this code out and have my favorite radar image source hammered over and over by everyone in the world. So basically… you need to find a site that will predictably show you their most current radar image for your area. Take a look at the page source and look for the line where the image is. As you can see above, I simply loop through every line on the page looking for “img src=”, because my particular source only shows one image on that page. I’ll have to leave it up to your imagination on how to scrape the image out of their code. Or, if the radar images don’t matter to you, you could skip this step altogether!

Incidentally, you might notice the variable ‘rightnow’. That is simply the current unix timestamp. I name my radar images using that variable so I always have a unique name for them.

if os.path.getsize(rightnow+”.gif”) > 0:
print “Got “+rightnow+”.gif”
c.execute(‘insert into weather values(now(),”‘ + stats[‘temperature_string’] + ‘”,”‘ + stats[‘temp_f’] + ‘”,”‘ + stats[‘relative_humidity’] + ‘”,”‘ + stats[‘pressure_in’] + ‘”,”‘ + stats[‘heat_index_f’] + ‘”,”‘ + stats[‘windchill_f’] + ‘”,”‘ + rightnow + ‘.gif”)’)
print rightnow+”.gif”+” Skipped”

And finally, I check to see if the image size is greater than 0 bytes. You’d be surprised how often empty images are saved. This just makes sure you’re not filling your database with useless information. Assuming it’s a good image, I insert the data from the NOAA’s XML page along with the name of the radar image I retrieved into the weather table.

And there you have it! Just create a cron job to run this script regularly (radar is generally updated every 6 minutes, but weather observations are only updated hourly) and watch this data accumulate in your new weather table. You can then go on to create neat graphs and other fancy things all based on your historic data. Enjoy!

a successful Polycom IP 501 deployment model

August 31st, 2006

Those who have embraced asterisk or other VoIP systems have a large choice of phones to deploy on the desks throughout their organization. I’ve personally ordered quite a few phones, and I’ve seen some that are very very bad from a user’s perspective. One particular model was horrible, to the extent of needing to be reset regularly. But one of the models I tested was actually recommended by a user on the asterisk IRC channel… the Polycom IP 501. It’s a multi-line SIP phone with all of the features you would expect to find in a solid desktop phone meant for business. They have a large screen, a very good full-duplex speakerphone, and a built-in switch… so you don’t have to run extra cabling.

Polycom IP 501One place they do fall short, however, is in the configuration. Many people actually shun Polycom because they refuse to provide reliable support unless you are an actual dealer. The configuration files aren’t well documented, and the firmware is fairly hard to find. That, my friend, is the reason I’m writing this post. I’m going to detail everything you need in order to get a quick deployment model setup in your company. By the end of this, you should be able to add a new phone to your network within about 15 minutes.

By the way, some of those who read this may question why I’m not even glancing in the direction of configuring the phones via the built-in webserver. I’ve taken this route for a few reasons… 1) the built-in webserver is hideously slow, 2) when you change the configuration enough, you have to wait for the phone to reboot (which takes a short eternity) before you can continue configuration, 3) it doesn’t make sense to do all of your configuration manually on every phone, especially if you have more than 2-3 users.

So now that we’ve made our decision, let’s start with the firmware. You can find the latest firmware and bootrom files right here. I suggest you get:


Just unzip those files onto your local drive, and try to keep from editing anything in that directory… just so you have clean files to reference.

Now is the time to install an FTP server. Unfortunately, as much as I hate these words, the details are beyond the scope of this article. The main point is that you’ll need a working FTP server that is accessible from your local network. Whether you decide to install it on a local Linux server following my instructions here, or you just install Filezilla Server on a Windows machine, you should be fine. On your FTP server, create a new user by the name of ‘PlcmSpIp’, case IS sensitive. One suggestion, when setting your password you should remember that you’ll be entering the password several times on a telephone keypad, so sticking to a fairly short numeric password might be a good idea. ;)

At this point, revisit the directory of files you unzipped. You’re going to need to copy the following files to your FTP user’s home directory:

  • bootrom.ld
  • bootrom.ver
  • sip.cfg
  • sip.ld
  • sip.ver
  • SoundPointIPWelcome.wav
  • The entire SoundPointIPLocalization directory

Also, you’ll need to create a directory called ‘logs’ in the user’s home directory. We’ll configure the phones to upload their log files to this location so that things are fairly tidy.

Take a look at the files you unzipped on your harddrive. You’ll find 000000000000.cfg, 000000000000-phone.cfg, and 000000000000-sip.cfg. These are the primary configuration files that will control every aspect of the way your phone behaves. We’re going to make these into template files, specifically for a program that we write to parse and create proper config files out of. Neat, eh? If you’d like to just use my template files, you can download them, or you can just use them for reference:

You’ll notice in each file I inserted various placemarks using the format “{ATTRIBUTE}”. These are in the 3 config files, and match up with the columns in the MySQL table. We’re going to create a Python script that will read in the data and create custom configs for each phone in the table. So let’s say we have the following information:

id server fname lname extension mac ringtone linekeys acdline acdlabel chirp
1 Joe Smith 1000 0a1b2c3d4e5f 4 2 2000 IT silence
2 Jane Doe 1001 0a1b2c3e4f5g 4 3 silence

So we’ve got the main server, the users’ names, their extensions and phone MAC addresses. Notice we also have the ‘linekeys’ column. The 501 has the ability to assign a line number to one or multiple lines on the phone. Since the 501 only has 3 physical line keys, I don’t want to mess with any more than that. As you can see, though, Joe is in the IT group. The idea is that all of your agents who are in queues will only want one line dedicated to that queue, that way Asterisk isn’t going to ring their phone if they’re already on a queue call. So by setting ‘linekeys’ to 2, and assigning an ‘acdline’ number and ‘acdlabel’, we set up the third line to be a separate extension number. Naturally, you’ll need to configure this properly in your Asterisk box. As far as Jane Doe goes, she just gets the same extension for all three lines, since she’s not a member of any queues.

Ok, so now we have our database set up with the user data, we’ve got template config files, and we have an FTP server set up. Only a few more steps to go. On the next page, we’re going to build the Python script that will tie it all together. I’ll see you there!

So to recap, we need to bring the FTP server, the configuration template files, and the phone user database all together into a fairly easy to use system. The way I did it was with a Python script. Is there anything Python can’t do? :)

Here’s the script:

import MySQLdb, os, shutil
print ” Connecting to Database”
db=MySQLdb.connect(host=”host”, user=”user”, passwd=”pass”, db=””)
c = db.cursor()

print ” Creating Company Directory”
output = open(“c:\\phones\\000000000000-directory.xml”,”w”)
c.execute(‘select fname,extension,ringtone from PBX.phones’)
for rec in c.fetchall():
fname,extension,ringtone = rec

print ” Creating CFG Files”
c.execute(‘select server,fname,lname,extension,mac,ringtone,linekeys,acdline,acdlabel,chirp from PBX.phones’)
for rec in c.fetchall():
server,fname,lname,extension,mac,ringtone,lines,acdline,acdlabel,chirp = rec
print ” ” + fname + ” ” + lname
password = “ae”+extension[::-1]
password2 = “ae”+acdline[::-1]
template = open(“c:\\phones\\000000000000.cfg”,”r”)
output = open(“Q:\\PlcmSpip\\”+mac+”.cfg”,”w”)
for line in template.readlines():
line = line.replace(“{SERVER}”,server)
line = line.replace(“{FNAME}”,fname)
line = line.replace(“{LNAME}”,lname)
line = line.replace(“{EXTENSION}”,str(extension))
line = line.replace(“{MAC}”,mac)
line = line.replace(“{LINES}”,str(lines))
line = line.replace(“{RINGTONE}”,str(ringtone))
line = line.replace(“{PASSWORD}”,password)
line = line.replace(“{PASSWORD2}”,password2)
line = line.replace(“{ACDLINE}”,str(acdline))
line = line.replace(“{ACDLABEL}”,str(acdlabel))
template = open(“c:\\phones\\000000000000-phone.cfg”,”r”)
output = open(“Q:\\PlcmSpip\\”+mac+”-phone.cfg”,”w”)
for line in template.readlines():
line = line.replace(“{SERVER}”,server)
line = line.replace(“{FNAME}”,fname)
line = line.replace(“{LNAME}”,lname)
line = line.replace(“{EXTENSION}”,str(extension))
line = line.replace(“{MAC}”,mac)
line = line.replace(“{LINES}”,str(lines))
line = line.replace(“{RINGTONE}”,str(ringtone))
line = line.replace(“{PASSWORD}”,password)
line = line.replace(“{PASSWORD2}”,password2)
line = line.replace(“{ACDLINE}”,str(acdline))
line = line.replace(“{ACDLABEL}”,str(acdlabel))
template = open(“c:\\phones\\000000000000-sip.cfg”,”r”)
output = open(“Q:\\PlcmSpip\\”+mac+”-sip.cfg”,”w”)
for line in template.readlines():
line = line.replace(“{CHIRP}”,str(chirp))


Remember, any time you use code from my site, you’ll need to remove the line numbers.

One thing you’re going to need to do is either have Samba installed on your local FTP server, or use something like NetDrive to map a remote FTP directory. Both options work great, though NetDrive would certainly be the slowest of the two. In any event, as you can see, I mapped my FTP directory to my Q: drive in Windows.

So the basic flow of this program is to first read in all of the users from the PBX.phones table and build a “directory” XML file. This is the file that the phone uses for the speed dials. If you’d like to leave it out, feel free. Then the script reads in the data again and goes through each template file, replacing the relevant placemarks with good data. The files are then saved to the FTP server using the MAC provided in the table.

At this point, we’re done. In all, it takes about 10-15 minutes to setup a brand new phone out of the box… and that includes the time the phone spends downloading the latest firmware. It’s an insanely easy process at this point to setup an entire office fairly quickly.

I hope this information is useful to someone out there. If so, please let me know!