Archive for September, 2006

drive a webcam with python

Sunday, September 17th, 2006

I bought a USB webcam off of eBay quite some time ago, and I decided to connect it to my telescope with a little bit of hardware hackery. I’ll have to see about posting a writeup on how I did that at a later time. Anyway, when I installed my camera software, I quickly found how horrible the program was. It gave a tiny preview of what the camera saw, and had no way of capturing images or video without waaaay too many clicks of the mouse. That’s when I decided to write my own in Python.

The main libraries that I ended up using were VideoCapture, PIL, and pygame. As you’ll see, it’s not really difficult at all to make a useful webcam application.

Here’s the code:

from VideoCapture import Device
import ImageDraw, sys, pygame, time
from pygame.locals import *
from PIL import ImageEnhance

res = (640,480)
pygame.init()
cam = Device()
cam.setResolution(res[0],res[1])
screen = pygame.display.set_mode((640,480))
pygame.display.set_caption('Webcam')
pygame.font.init()
font = pygame.font.SysFont("Courier",11)

def disp(phrase,loc):
    s = font.render(phrase, True, (200,200,200))
    sh = font.render(phrase, True, (50,50,50))
    screen.blit(sh, (loc[0]+1,loc[1]+1))
    screen.blit(s, loc)

brightness = 1.0
contrast = 1.0
shots = 0

while 1:
    camshot = ImageEnhance.Brightness(cam.getImage()).enhance(brightness)
    camshot = ImageEnhance.Contrast(camshot).enhance(contrast)
    for event in pygame.event.get():
        if event.type == pygame.QUIT: sys.exit()
    keyinput = pygame.key.get_pressed()
    if keyinput[K_1]: brightness -= .1
    if keyinput[K_2]: brightness += .1
    if keyinput[K_3]: contrast -= .1
    if keyinput[K_4]: contrast += .1
    if keyinput[K_q]: cam.displayCapturePinProperties()
    if keyinput[K_w]: cam.displayCaptureFilterProperties()
    if keyinput[K_s]:
        filename = str(time.time()) + ".jpg"
        cam.saveSnapshot(filename, quality=80, timestamp=0)
        shots += 1
    camshot = pygame.image.frombuffer(camshot.tostring(), res, "RGB")
    screen.blit(camshot, (0,0))
    disp("S:" + str(shots), (10,4))
    disp("B:" + str(brightness), (10,16))
    disp("C:" + str(contrast), (10,28))
    pygame.display.flip()

I decided to use pygame in order to build this because it can actually handle the fps that I need for video. I built something similar quite a while ago with wxPython, but it just wasn’t fast enough at image swapping to make it work well.

A couple of noteworthy points:

  • The function on line 15 is simply there to help automate displaying information on the screen. In my case, I display the number of screenshots I’ve saved to disk, and the current brightness and contrast levels. The function displays the text sent to it by writing it in dark gray, and then re-writing it in light gray over the top with a pixel offset. I found this helps a TON to make the text visible in different scenarios.
  • Line 26 is where I actually apply the contrast and brightness changes. Lines 30-33 set up the number keys 1-4 to adjust contrast/brightness on the fly. This is terribly useful, in my experience.
  • Lines 34 and 35 are setting up ‘q’ and ‘w’ to show the settings dialogs for the webcam… this is where you can set resolutions, exposure levels, etc.
  • On line 36, and those under the if statement, I save the current image and name it using the current time… this insures no overlapping image names.

If you’re trying to write a webcam app of your own, I hope this gets you pointed in the right direction. Let me know if you find this info useful!

get it : PhotoRec

Thursday, September 7th, 2006

It’s funny how often I find myself deleting things. I mean, Shift-Delete is hard-wired in my brain, just because I delete constantly and hate it when my Recycle Bin sits there full. But the worst thing of all is when I go in to innocently delete a directory that I deem useless, and I remember days after the fact that there was something important in it. That’s why PhotoRec is such a life saver. Before you prejudge it by it’s name, this is a real file recovery program… it’s not just for photos. The list of recognized file types is pretty huge, in fact.

Basically, PhotoRec uses data carving techniques in order to find the files. Data carving is the process of extracting a collection of data from a larger data set… files are “carved” from the unallocated space on the drive. The nice thing is, it works independently of the filesystem headers, so it’s possible to recover files even if the partition info has been screwed up. The negative side of it all is that quite often data carving results in a lot of false positives. So tbh, you’ll end up with a directory of tons of files, and you’ll need to run through that directory in order to find the files that are actually good. But hey, since the purpose is to recover lost files, at least it sides on the ‘false positive’ side rather than the ‘false negative’, right?

Anyway, it’s terribly useful, and I’ve personally been able to recover many lost files with it. I hope your results are just as good!

install Pure-FTPd on Ubuntu

Tuesday, September 5th, 2006

I’m sure these instructions are similar for most distributions, but I thought I’d specifically target Ubuntu so that I don’t have to cover the little differences that each and every person might run into. The end result of this is to have a fast and secure FTP server running on Linux that, when given the proper credentials, will serve up a user’s home directory. You can imagine when this would be useful. I decided to go with Pure-FTPd for quite a few reasons, but mainly because of solid security and many configuration options. I’ve gotten myself into trouble before by choosing very simple FTPd servers and realizing later that they just weren’t able to do what I wanted them to do. Pure-FTPd is different that way.

So to get started, in Ubuntu just use apt to get Pure-FTPd installed:

apt-get install pure-ftpd

Now, set up a new user, and open up /etc/passwd for editing. Add /./ to their home directory like this:

/home/username/./

This keeps them from seeing the filesystem.

Also, make sure to put in a shell at the end of that line… /bin/bash works fine. After all of that, add their username to /etc/ftpallow.

Now it’s time to give Pure-FTPd a basic configuration. Edit /etc/pam.d/pure-ftpd to look like this:

auth sufficient pam_ftp.so
auth required pam_unix_auth.so shadow use_first_pass
auth required pam_listfile.so item=user sense=deny file=/etc/ftpusers onerr=succeed
auth required pam_listfile.so item=user sense=allow file=/etc/ftpallow onerr=fail
auth required pam_shells.so

Now just type pure-ftpd-control restart on the command line and you’re all finished!

chirp chirp chirp

Tuesday, September 5th, 2006

Have you ever been sitting there, happily chatting away with some friends, and someone makes a joke that’s well… it’s just SO unfunny that you have to point it out? A joke that is meant to be hilarious, but bombed so hard that everyone is just left looking at the ground or at each other, wondering what on earth just happened? It’s occurred quite a few times around me, and every time I can’t help but think of those same wonderful moments alluded to on the Simpsons, where you hear the wind blowing through the character’s hair, or hear a wolf howling, or (my personal favorite) hear a cricket chirping.

That’s why I set out to build a program that sits in your system tray, anxiously awaiting the moment when you click on it and it gets to chirp it’s little heart out. So without further rambling, I present Cricket 1.0. Pretty simple… just download it and run it. You’ll see a little icon in your tray that you can double-click at will.

Go ahead! Make fun of your coworker’s unfunny jokes! It’s quite possibly one of the most enjoyable things ever. ;)

Have fun, and if you find this program useful, please comment! Also, if you would like the source code, just ask. I’m not posting it right now just because I figured no one would really want it… but if you’d like it for reference or modification, I would be happy to provide it.

get it : ToDoList

Tuesday, September 5th, 2006

I just ran across a really great todo list program via Digg, and thought I’d mention it. It’s got a TON of functions that are all really really useful, plus it’s free and open-source. I’ve needed a program that would allow me to manage all of my projects and sub-projects at once, and this one fits the bill perfectly. w00t! :)

use macros in asterisk

Monday, September 4th, 2006

I was just watching an episode of Systm the other day with a friend… Episode 5, the one where they describe how to build a basic Asterisk server. First, I’ve gotta say, I love that Kevin Rose and the other guys over there have created Systm… it’s a very cool idea, and something that honestly just sounds like a blast to do.

Anyway, while watching it, I remembered back when I was learning asterisk that many would-be mentors simply showed how to create a static definition in extensions.conf for every extension. While that works for a small system, it quickly becomes needlessly complicated when you need to change the way things flow for every extension after that. Just imagine changing something as small as the voicemail context for 50 extensions. It just doesn’t make sense. That’s where macros come in. They make it possible to cover most of the extensions with one script. The issue, I’ve found, is that most example macros just aren’t useful enough. :)

For my users, I want to give them the ability to call their own number in order to go straight to voicemail. That, and the Polycom 501 actually defaults to calling itself when you hit the ‘messages’ button. So we need to create a macro that will perform the normal functions, but will also know when you’re calling yourself.

First, in the main context you’ll need a line like this:

exten => _10XX,1,Macro(sipphone,${EXTEN})

This assumes extensions between 1000 and 1099. Change it to suit your needs. The point of this line is that when someone dials a ’1000′ extension, it will send the call to the ‘sipphone’ macro, sending the actual extension dialed along with it.

Then, somewhere else in the file, perhaps just above your main context, add:

[macro-sipphone]

exten => s,1,GotoIf($["${ARG1}" = "${CALLERIDNUM}"]?5:2)
exten => s,2,Dial(SIP/${ARG1},20)
exten => s,3,Voicemail(su${ARG1})
exten => s,4,Hangup
exten => s,5,VoicemailMain(${CALLERIDNUM})
exten => s,6,Hangup

So when a call is sent to this macro, ${ARG} would equal the extension dialed, and ${CALLERIDNUM} would be the extension dialing. On the first line of the macro, we’re simply saying “if the caller id and dialed number are equal, send the call to line 5, or voicemail. Otherwise, we follow the normal flow of a macro and ring the extension for 20 seconds, and then send the call to voicemail. Also, notice that if you call yourself, we use the VoicemailMain command instead of the Voicemail command… that sends you in to listen rather than record.

making a radar loop with PHP

Sunday, September 3rd, 2006

So a few days ago I was out with friends and a storm was on it’s way in. Naturally, I’m the one they ask whether it’s ‘about to hit us’, so I pull up the web browser on my Treo 650 and realize… um… there’s no quick way to pull up a radar loop. I mean, I used the great mobile site from the NWS, but it still didn’t give me the effect I wanted. And hey, I’m already filling a database with weather observations and radar images! So I decided to see what it would take to make a custom radar loop that I could get to at any given moment. The Treo can display animated gif files, so that seemed like the most efficient solution. The question is, how do we create an animated gif using PHP?

Well, I don’t like to reinvent the wheel, especially if someone else out there has made a wheel superior to one that I could make myself. :) Hence, I found this. From here, we just need to read in the images and feed them to the GifMerge class.

[php]
Header(‘Content-type: image/gif’);
require(‘GifMerge.class.php’);

$count = 1;
$q = mysql_query(‘select * from (select recorded, radar from weather order by recorded desc limit 10) as t order by recorded’);
while($r = mysql_fetch_array($q)){
$image_p = imagecreate(300, 300);
$image = imagecreatefromgif(“radar/”.substr($r['radar'], 35));
imagecopyresampled($image_p, $image, 0, 0, 210, 120, 300, 300, 300, 300);
imagegif($image_p, “radar/loop/frame”.$count.”.gif”);
$count++;
}

$__i = array(“radar/loop/frame1.gif”, “radar/loop/frame2.gif”, “radar/loop/frame3.gif”, “radar/loop/frame4.gif”, “radar/loop/frame5.gif”, “radar/loop/frame6.gif”, “radar/loop/frame7.gif”, “radar/loop/frame8.gif”, “radar/loop/frame9.gif”, “radar/loop/frame10.gif”);
$__d = array(30, 30, 30, 30, 30, 30, 30, 30, 30, 300);
$gifmerge = new GifMerge($__i, 1, 1, 1, -10, $__d, 2, 0, 0, ‘C_FILE’);
echo $gifmerge -> getAnimation();
[/php]

One issue I had was that the images I scrape are rather large, and when you merge 10 of them into an animated GIF, you end up with an image that a little bigger than most handheld devices like to deal with. Not to mention, we’re talking about a GPRS connection here… it would be easier to download 50kb than 500kb. So to get around that, I used the imagecopyresampled function to cut out the portion of the image most important to me. I then save the cropped image into a ‘loop’ directory as 10 separate ‘frame’ files. After that, it’s pretty straightforward to follow the GifMerge example to create the image.

So without further ado, here’s the result:

filling a database with historic weather data

Saturday, September 2nd, 2006

I’m one of those people that just loves inclement weather. Maybe it’s because of where I grew up, but I’ve always adored a good thunderstorm. Whenever a storm is heading for my area, I check various sites online to get the latest radar and what-not, but sometimes you just aren’t near a computer (gasp!). Not only that, but most of radar loops you find only go back so far… maybe an hour or two. That’s why I decided to build my own solution, using data that the NOAA provides for free. Pretty much everything here is written in Python, the greatest language ever. :)

So first of all, I had to decide how to actually collect the data. There are a few methods, but one issue I faced is that I normally look at this stuff on my laptop. What if I have my laptop turned off? It’s just not a reliable platform to use for mining data. Happily, I have a dreamy webhost that gives me shell access. So there you go… the perfect spot to house the fruitage of my scripts. So what I’m going to do is build a script that runs regularly, contacts a weather site to scrape a radar image, and contacts the NOAA to get the latest weather data… temperature, barometer, etc. At that point, I want to insert this stuff into a database housed at the same host.

I’m going to take you through this as if you’re fairly new to Python, and the plan is to be really easy to understand. If it isn’t… well… you get what you pay for, right? :) Seriously though, if you end up with questions, just post them at the end of this article and I’ll answer them.

So I logged into my shell account at my webhost via SSH. I created a new text file called ‘radardl’ using the command vi radardl, and then made it executable with chmod +x radardl. As for the code that goes in that file…

[python]
#!/usr/bin/python2.4
import MySQLdb, time, os
from urllib import urlopen, urlretrieve
[/python]

Nothing special here… just set it up to execute using your Python executable, and import the necessary modules. MySQLdb is used for working with the database, and urllib is used to get the web pages / images that we’ll need.

[python]
db = MySQLdb.connect(host=’dbhost’, user=’dbuser’, passwd=’dbpasswd’, db=’dbname’)
c = db.cursor()
[/python]

This is the basic connection to your database. Obviously, you’ll need to set one up somewhere so that you’ll actually be able to do this. Replace the placeholders with your specific connection info. You’ll also need to create a new table to hold your data… I used something like, “create table weather (recorded timestamp primary key, temperature_string varchar(50), temp_f float, relative_humidity float, pressure_in float, heat_index_f float, windchill_f float, radar varchar(250))“. For simplicity, go ahead and use that command to create your table and you can modify it after we’re done.

[python]
j = urlopen(‘http://www.weather.gov/data/current_obs/KTUL.xml’)
[/python]

Ok, this is where we download the current weather observations. Take a look at this page and select your state, then find the weather station closest to you. For me, it’s KTUL in Oklahoma. Just find your local page, and replace the url in the code. While you’re looking at it, take a look at all of the data that’s provided in there. It’s a wealth of observation data! I’m only capturing a couple of things with this code, but you can easily add what you want.

[python]
stats = {}
for l in j.readlines():
stats[l[l.find("<")+1:l.find(">")]] = l[l.find(">")+1:l.rfind("<")]
[/python]

Now, you could use something fancy like the wonderful Elementtree library, but in my case my host doesn’t have it installed and I couldn’t do it myself. So instead I used the above code to give me a similar result. Basically, I search for the < and > characters in different ways in order to get the key name and the value. I then store those into the ‘stats’ array. This works great for the XML that the NOAA provides… but keep in mind that XML from other sources may not be so easy to parse.

[python]
rightnow = str(time.time())
j = urlopen(“[INSERT WEBSITE OF YOUR CHOICE HERE]“)

for line in j.readlines():
if line.find(“img src=”) > -1:
img = line.strip()[11:-1]
urlretrieve(img, rightnow+”.gif”)
[/python]

Ok… the thing is, everyone has their favorite weather radar, and I don’t want to advocate one over the other. I’d also prefer not to give this code out and have my favorite radar image source hammered over and over by everyone in the world. So basically… you need to find a site that will predictably show you their most current radar image for your area. Take a look at the page source and look for the line where the image is. As you can see above, I simply loop through every line on the page looking for “img src=”, because my particular source only shows one image on that page. I’ll have to leave it up to your imagination on how to scrape the image out of their code. Or, if the radar images don’t matter to you, you could skip this step altogether!

Incidentally, you might notice the variable ‘rightnow’. That is simply the current unix timestamp. I name my radar images using that variable so I always have a unique name for them.

[python]
if os.path.getsize(rightnow+”.gif”) > 0:
print “Got “+rightnow+”.gif”
c.execute(‘insert into weather values(now(),”‘ + stats['temperature_string'] + ‘”,”‘ + stats['temp_f'] + ‘”,”‘ + stats['relative_humidity'] + ‘”,”‘ + stats['pressure_in'] + ‘”,”‘ + stats['heat_index_f'] + ‘”,”‘ + stats['windchill_f'] + ‘”,”‘ + rightnow + ‘.gif”)’)
else:
print rightnow+”.gif”+” Skipped”
os.unlink(“suckfreemusic.com/radar/”+rightnow+”.gif”)
[/python]

And finally, I check to see if the image size is greater than 0 bytes. You’d be surprised how often empty images are saved. This just makes sure you’re not filling your database with useless information. Assuming it’s a good image, I insert the data from the NOAA’s XML page along with the name of the radar image I retrieved into the weather table.

And there you have it! Just create a cron job to run this script regularly (radar is generally updated every 6 minutes, but weather observations are only updated hourly) and watch this data accumulate in your new weather table. You can then go on to create neat graphs and other fancy things all based on your historic data. Enjoy!