We're full of it . . .

Accessibility, standards, usability, ideas, coffee, bells & whistles, knowledge, and great design!

Mug

Ruby Home Automation for Canine Happiness

Published By Corey on Feb.13.2010 at 9:30AM

A problem presents itself...

Part of the wonderful part of working from home is being able to spend lots of time with our 2 Boston Terriers Molly and Daisy. However, my office is in the upper floor of the house I have to keep tabs on when they might need to go outside to "make potty".

They're smart enough to go to the door, scratch it and wait but occasionally (usually when over-focused on some piece of code) I forget and they're forced to make use or the floor in the washroom.

In the spring and fall months I would end up just leaving the door to the backyard open for them to come and go as they please. In the summer the hydro bill skyrockets with AC usage not to mention the mosquitoes and other unwelcomed bugs that find their way inside. In the winter the furnace has a tough time keeping up not to mention the wasted energy.

Envisioning a solution

I started thinking of how a video camera of some kind would enable me to watch them from a secondary screen like my iPhone but wait second all these new home security cameras come with motion notification, why couldn't I just tap into that?

The puzzle pieces

A Linksys WVC54GC Wireless-G Camera for $55 at Canada Computers
FFMpeg installed via DarwinPorts:sudo port install ffmpeg +gpl +postproc +lame +theora +faac +faad +x264 +a52 +xvid
ImageMagick installed via DarwinPorts:sudo port install ImageMagick
Growl, why bother with emails when I can have a slick notification popup on any number of computers
A 70 line ruby script to glue it all together.

How does it all work?

Framegrabbing the stream with FFmpeg

The first piece is just connecting to the video stream (SOURCE variable) and framegrabbing every couple of seconds (INTERVAL variable). A simple ffmpeg command will do this and keep the frames numbered
ffmpeg -i http://172.16.187.5/img/mjpeg.cgi -y -ss 2 -an -sameq -f image2 -r 1/2 frames/%07d.jpg
Note the rate is in FPS so we want 0.5 fps or 2 seconds per frame.I use IO.popen to start a child process that will exit when the script quits. The only issue is for some reason occasionally the ffmpeg process dies off. I'm still trying to figure out why this happens....

Watch for new frames

The directory_watcher gem does has a simple event interface for new files. We can even tell it to poll at the same rate we expect frame grabs to come in. When a new frame shows up we run the comparison routine to see if anything has changed.

Compare the frames

This was the hardest part. I did some heavy googling I couldn't really find much in the way of motion detection algorithms, but after looking at the ImageMagick documentation I stumbled upon compare and everything came together. As much as some cool highlighting or rectangles of change would be great i really just need to know if the image changed enough to suspect the dog might be at the door. I can look at the image to verify if it is a false positive


Frame 001: Nothing going on

Frame 002: Nothing to see here, move along people
We can compare the last and current frames to see whats changed. Since the stream is using lossy jpgs there will be differences on almost every pixel each frame if we do a straight up comparison with
compare -fuzz 0% -metric AE ha_001.jpg ha_002.jpg ha_comp_fuzz0.jpg
==> 68047

68047 of the 76800 pixels changed according to compare, that is 88.6% of the image!

I'm guessing red is bad...
The real changes look very negligible if we enable some fuzz
compare -fuzz 10% -metric AE ha_001.jpg ha_002.jpg ha_comp_fuzz10.jpg
=> 233

Here we have 233 pixels of change meaning only 0.3% of the image changed. I find that 10% fuzz works well enough, in high daylight conditions 5% wasn't enough because just the motion of the clouds caused the lighting in the room to change quickly.

With fuzz the changes are minimal

What happens when a dog is in the frame?


Frame 002: The emptiness

Frame 003: Hello puppy!
When we use the compare utility against a frame with a dog in it we have plenty of difference even with fuzz:
compare -fuzz 10% -metric AE ha_002.jpg ha_003.jpg ha_comp_event.jpg
=> 20189

In this case we see a fairly large change in the image (26%) so its time to notify me.

We can clearly see an event has occured

Notifying when there is an event

Sending a growl notification when the image changes is pretty simple, we just use the growl event and attach the current frame to the notification so I can see it. The growl gem has a simple syntax and handles all the dirty work. I use the Music Video style for the notification so the image is as large as possible and make the notification sticky so if I'm not looking at the screen the instant it happens it stays around.

Conclusion

Lower heating bills, dogs that aren't nervous about having accidents in the house, damn thats not bad for an hours work. Writing this blog article probably took about as much time as putting the solution together once I found ImageMagick's awesome compare tool. If you have any questions or comments just drop me a note on Twitter!