Roomba goes WiFi

Last Tuesday I received a really cool device, the Roomba Wi-Fi Remote. In short: RooWifi.
The RooWifi turns your Roomba Robot Vacuum cleaner into a Wifi-enabled device!

RooWifi board with cover

Some specs:

Physical

  • Mini-DIN 7 pins (PS/2) plug
  • PCB dimensions: 53 x 32.5 mm.

WLAN

  • Wi-Fi IEEE 802.11b
  • Range up to 400m, Baud rate: 1 and 2 Mbps, AES128 encryption, WPA(2)-PSK, Hotspot and ‘Home network’ mode

Interfaces

  • Web Server with authentication.
  • TCP Socket Server for client-server applications

Protocols and Standards:

  • XML: Read-only with data refresh of 500ms.
  • JSON: Read-only with data refresh of 500ms.
  • AJAX Request through GET Calls and text format output.

Operating Systems, Compatible devices:

  • Android >=2.0
  • iOs >=3.1
  • Blackberry
  • WebOs
  • Windows Phone
  • Linux
  • MacOs
  • Windows.

Power consumption: 200mA @ Roomba Battery Voltage (around 17V).

We have a Roomba 500-series vacuum cleaner in our house since 2010. It helps us to keep the floors clean, cause with 3 cats running around things can get messy really quick (oh, and don’t forget the kids, they’re good at making a mess too ūüėČ
I control the Roomba with an IRTrans Ethernet module that’s located in the living room; I can send a ‘CLEAN’ command to the Roomba, but that’s about it – the DOCK command doesn’t work, and every now and then the IR command doesn’t work at all (I think due to too much sun reflections inside).¬†So controlling the Roomba has always been far from perfect.

I’ve been looking at other solutions to control the Roomba in the past, like attaching an Arduino to it but all the solutions I found resulted in a big ‘bulb’ on top of the Roomba, which I didn’t like. But this RooWifi is small enough!

All the Roomba’s (I know about) have a 7-pin mini-DIN SCI (Serial Command Interface) socket somewhere. And there’s even (official) documentation about this SCI port.

To control the Roomba, this SCI has to be accessible; where the socket is and how easy the physical access to the socket is differs per model. To get access to the SCI socket of my Roomba 563-PET I had to remove the top plate. After removing the battery and using a screw driver the top plate came off relatively easy. This plate will not be needed anymore. The RooWifi module can be inserted into the socket like this:

RooWifi connected to Roomba

Other models may need a different approach.

RooWifi with coverAfter plugging in the RooWifi into the SCI socket the RooWifi starts up in Hotspot mode and creates a ROOMBA WR wireless network.

I connected to it with a laptop which got an IP address of 10.0.0.2, opened a web browser to and found the Web Interface. From there you can alter the settings of the RooWifi so that it connects to your own Home Wifi Network if you want to.

Below you see a screen-dump of the Driver’s Remote:

RooWifi Drivers's Remote

Other screens show four main buttons for operating the Roomba (CLEAN, SPOT, DOCK, IDLE) and an overview of all the sensor values:

Roomba sensor values

Whooaahh, and all in real-time of course, really cool ūüėČ

It was fun driving the Roomba through the living room, the whole family was suddenly very interested in a test-drive, but that’s not why I bought this RooWifi of course; I’ve got other toys for that. The RooWifi is for automating the cleaning process, integration of the Roomba in the rest of my Home Automation system is cool, cause that means I have full control over when and for how long the Roomba will do its job – and with the Wifi connection we can use one single Roomba for more rooms than just the living room, cause Wifi is ‘everywhere’, and Infrared is not…

By the time I’ve finished the software for controlling and monitoring the Roomba I’ll write a follow-up with all the details like working with the Remote TCP server, the JSON data and all the other neat stuff the RooWifi has to offer.. so stay tuned!

The Raspberry Pi Camera Board

More for fun than anything else, I bought a Raspberry Pi Camera Board¬†about 10 days ago. Yesterday it arrived. It’s time to play ūüėČ

The first thing I did was connecting the Camera Board to one of my RPi’s. It’s relatively easy to do. The connector that has to be used is the one between the HDMI- and Ethernet-connector. Gently pull it up and insert the cable so that the silver-colored side is facing the HDMI connector. When the cable is inserted, push down the connector again. (there’s a video here explaining it all in more detail).

I started with a fresh SD card (I have to start labeling them!) and opened an SSH session to the RPi. The following sequence of commands are necessary to get your RPi into shape (i.e. fully upgraded) so that it can handle the Camera Board:

sudo apt-get update

This will synchronize the package index files on your RPi, so that once this command has completed, your RPi ‘knows’ all about the available packages. This is necessary to perform the next command:

sudo apt-get upgrade

This will install the newest versions of all packages currently installed on the RPi, based on the information retrieved with the previous command. You can get yourself a cup of coffee now, this will take a while ūüėČ

sudo raspi-config

To enable camera support, which is not enabled by default, you’ have to run raspi-config, enable camera support (menu option available in the main menu) and do a reboot after that.

From here you can use the Camera Board, capture jpeg images, videos and so on. However, assuming that would all work out of the box and because I wanted to stream video to my PC, I also installed VLC:

sudo apt-get install vlc

OK, all is still going smooth… I found a nice How-To that helped me to get raspivid & cvlc produce a video steam on port 8090 of the RPi:

raspivid -o – -t 99999999999 -hf -fps 25|cvlc -vvv stream:///dev/stdin –sout ‘#standard{access=http,mux=ts,dst=:8090}’ :demux=h264

Now all that was left to do was starting VLC player on my PC, tell VLC where to get the video stream (http://<RPi-IP-address>:8090) and there it was:

VLC snapshot

The image above was created with the snapshot-feature of the VLC player; clicking the image will show the full-res HD snapshot of 1920 x 1080 pixels, converted to jpeg to reduce the file size a bit. Not bad… I mean, I’ve seen worse on >100 Euro IP cameras.

So, what’s next? Setting up the Camera Board was easy and I like what I see, so what am I going to use it for… cause this camera board is to neat to end up on a shelf! But first I have a couple of things I want to explore further. For instance, I would like to have a somewhat longer ribbon cable between the board and the RPi; how to reduce the lag in the video stream; and I also want to be able to control the Rpi/Camera remotely (start/stop streaming, taking snapshots triggered by whatever goes on in and around our house, etcetera); adding PT (Pan & Tilt) control to the camera would be nice too, like I did some years ago with Arduino & XBee modules. Easy access to the images and captured video from my network would be handy too. Oh, and a time-lapse video is something I’d also like to try.

But before all that, I first need to find a way to protect the Camera Board from wet cat noses, cause with 3 curious cats running around in our house I know that leaving it unprotected will result in a short life of the Camera Board. In short: enough things to keep me busy for quite some time!

Backing up Raspberry Pi to Synology NAS

With what could have happened 2 weeks ago if I wouldn’t have had a good backup plan, I realized that I had to do something about those Raspberry Pi’s that have started invading our house and how I’m backing up the code running on those small Linux computers.

Cause what happens in practice is that you open a telnet/ssh session on the RPi, start coding, debugging & testing until it works. And by the time you’re finished with that and made sure that everything starts automatically after a reboot etcetera, you close the session and you’re done – on to the next adventure; right? Wrong.

This leaves all the code on the RPi, on a 4GB SD card with no backup at all… sounds dangerous! I just had to do something about that, before I would lose something.

The first thing that popped up in my mind was rsync, a tool to sync files and directories to another machine. And since I have 2 Synology NAS devices, I thought it would be smart to backup my RPi’s to one of those instead of making images of the SD cards. Cause the latter option would result in downtime each time I want to do a backup; yuck. So rsync it will be.

All the examples I found that worked with rsync made use of ssh, so that was the first hurdle I had to take – I’ve never done that much with public & private keys, key generators and so on. But with some help I managed to get things going. First you have to generate a key with ‘ssh-keygen -t rsa‘ on the NAS. The resulting file with the public key (default file name id_rsa.pub) has to be added to the authorized_keys file on the RPi. Normally you should be able to use ssh-copy-id command for that, but for some reason that didn’t work on my NAS, so I had to manually copy the file to the RPi and do a ‘cat id_rsa.pub >> authorized_keys’ to make that happen.

After that I could ssh from my NAS to the RPi without the need of entering a password (which is needed for the script to run unattended!).

Ok, next item on the list was making a rsync script.

After some tries I came up with this:

/usr/syno/bin/rsync -avz --delete --exclude-from=/volume1/backup/_scripts/rsync-exclude.txt -e "ssh -p 22" root@raspdev.hekkers.lan:/ /volume1/backup/raspdev/ >> /volume1/backup/raspdev_backup.log 2>&1

And it worked! In about 10 minutes the rsync script completed its work:

RPi backup on Synology

Last but not least, scheduling the backup job. It seems that since DSM 4.2 the Synology Task Scheduler has an option to run user-defined scripts. Just what I need:

Synology Task Scheduler

Cool.. so now my ‘master’ Synology NAS (a DS209) takes care of backing up the RPi and 1 hour later a Network Backup job makes a copy to my other NAS (DS109) ; so now I have 3 versions – the original and 2 copies. Try to beat that, Mr. Murphy! ūüėČ

It’s roughly the same way how I backup my Windows servers – robocopy copies the important directories to the first NAS and Network Backup does the rest.

Now some statistics on how rsync is performing. Here’s a part of the initial rsync backup log:

sent 972675 bytes
received 1363869597 bytes
2347106.23 bytes/sec
total size is 1360002151 speedup is 1.00

It does take some processing power to perform the back up though:

Backup process load

70% CPU for process-id’s 3914 and 3921 (3rd and 4th line), that’s quite a lot. Removing the ‘z’ option (=compression during transfer) from the rsync command resulted in lower CPU usage and I might try to make the backup somewhat ‘nicer’ to reduce the CPU usage even more – backup is not that time critical but other processes running on the RPi might be… I’ll have to do some tests to see what’s best practice.

Daemonizing drivers, Python, MQTT and web scraping

I’ve been doing some excursions¬†during the last 12 months to figure out what would be the best way to replace my Windows-based Home Automation system. It has become rather large through the years and I want to get rid of that single point of failure, being that Windows executable that does it all. A few weeks ago I tried some things with a Raspberry Pi¬†(RPi) and Python and was actually very much surprised by the speed with which I could build something from scratch in a matter of hours.

FEZ Panda II, Simplecortex are both nice and may be able to do the job just as well, but the RPi is just awesome compared to those 2 for ¬†reasons I think I don’t have to explain.

Yesterday afternoon was the first time (after a long, long week full of all kinds of problems) that I could sit down behind my PC again and spend some time on things I like to do. That afternoon I tried to get a Python script to run as a daemon. Running a Python script from the command-line is not very useful for me (only for testing & debugging) cause what I really want is that the RPi starts all the tasks I want it to run on it automatically at boot time Рjust power-up the RPi and the rest goes automagically, no user interaction should be needed. So I decided to combine 3 subjects into this small experiment: a Python MQTT client, daemonizing Python scripts and web scraping; lets see how far I can get.

I started with a brand new RPi, SD card and new image written to it. Did the usual things like¬†setting the hostname, time zone etcetera. After that I installed some Python¬†related tools (e.g. installing modules)¬†to make life easier. An MQTT client¬†written in Python wasn’t very hard to find – Mosquitto (the OS MQTT server that¬†I’m using since September last year) has one, so I installed that one too. I searched for info on how to create a daemon and found the python-daemon library.¬†The last things I needed were some tools to make the process of web scraping somewhat more comfortable:¬†regular expressions and requests. I found some examples on how to¬†use all these libraries/tools mentioned above and started coding.

Well, is this something you can really call coding?¬†Reading and understanding the examples I found, copying code snippets, adding some extra lines,¬†deleting others – it feels more like blending actually ūüėČ

I use web scraping to show various kinds of information on the User interfaces in our house –¬†things like the amount and total length of traffic jams, a 2-hour rain forecast (BBQ!)¬†for our specific location and stuff like that, so it looked like a good idea to¬†have a look if I could make one of those scrapers run in a Python-based daemon on the¬†RPi.

3 hours later I was finished, with this code as a result:

import logging
import time
import mosquitto
import requests
import re
from daemon import runner

class App():

    def __init__(self):
        self.stdin_path = '/dev/null'
        self.stdout_path = '/dev/tty'
        self.stderr_path = '/dev/tty'
        self.pidfile_path =  '/var/run/webfetcherd.pid'
        self.pidfile_timeout = 5

    def on_connect(self, mosq, obj, rc):
        logger.info("on_connect:"+str(rc))

    def on_message(self, mosq, obj, msg):
        logger.info(msg.topic+" "+msg.payload)

    def de_html(self, html):
        pattern = re.compile("<.*?>|&nbsp;|&amp;",re.DOTALL|re.M)
        return pattern.sub("",html)

    def run(self):
        while True:
            self.mqttc = mosquitto.Mosquitto("webfetcher")
            self.mqttc.on_message = self.on_message
            self.mqttc.on_connect = self.on_connect
            self.mqttc.connect("192.168.10.40", 1883, 60)
            rc = 0
            prvtime = 0.0
            while rc == 0:
                rc = self.mqttc.loop()
                if (time.time()-prvtime) > 60:
                    prvtime = time.time()
                    rq = requests.get('http://m.fileindex.nl/files.js')
                    rex = re.compile('"(.*)"')
                    m = rex.search(rq.text)
                    if m:
                        res = self.de_html(m.group())
                        logger.info('Match: %s', res)
                        self.mqttc.publish("test/trafficjams",str(res))
                    else:
                        logger.info('NO match: %s', rq.text)

logger = logging.getLogger("Webfetcher")
logger.setLevel(logging.INFO)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(messa
handler = logging.FileHandler("./webfetcher.log")
handler.setFormatter(formatter)
logger.addHandler(handler)
app = App()
daemon_runner = runner.DaemonRunner(app)
daemon_runner.daemon_context.files_preserve=[handler.stream]
daemon_runner.do_action()

Starting the daemon
Done! Ok, not completely – I didn’t make an init script yet and I don’t really like the way the MQTT client is called (line 36) and there are some other things that can probably be done better, especially when there’s I/O involved with hardware connected to the RPi; but for this goal and as result for a first alpha version this is OK I guess..

On to alpha2!

Audyssey rocks

About 10 days ago I visited someone who takes home cinema serious – very serious! He built a cinema under his house, roughly the size of our living-room; what I saw and heard there was astonishing, overwhelming and amazing in every way!

During the demonstration of his Home Cinema he also showed some of the things he had done to improve the performance of his audio & video equipment and 2 things were very clear РAudyssey and the Darbee Darblet were the keywords.

I was particularly struck by the influence that Audyssey can have on the sound. Wait, wait… Audyssey, doesn’t my Onkyo TX-NR709 also have that on board? Yes it does! So around midnight when I left on my way back home in my car, I was already making plans for the next evening – I’m gonna hook up the Audyssey mic to my AV receiver and do a Audyssey run!

Audyssey mic on a tripod

So that next evening I managed to get everyone else upstairs around 10 o’clock (“you’re all looking very tired”)¬†and I was ready to perform my 1st Audyssey run. I was¬†curious if I would witness that same big improvement in sound quality as I did the day before, where I heard the difference between sound with and without Audyssey influence.

Half an hour later the Audyssey test was completed – and WOW, I just couldn’t believe it, but my ears didn’t lie – everything¬†had changed! All surround speakers now performed equally well which gave a tremendous boost to the ‘surround’ experience, the highs and the lows were all present in the way they should be – well, I just can’t describe it but this was really a huge improvement! The next day the rest of the family was just as surprised by the new sound as me – unbelievable..

So why didn’t I do this when I started using this AV receiver? Stupid me… I never thought it would make such a big difference and after a month or so the Audyssey mic disappeared in a box in the garage and I never thought about it again. But it only takes about half an hour or so and the impact is really worth it. Even if it would take a whole afternoon, now that I know the end result, I would always¬†do a Audyssey optimization from now on.

So if you also have a AV receiver with Audyssey mentioned on the casing or mentioned in the manual – use it, it’s Highly recommended!

Oops, 1 TB at risk

read errors

This is what happened last Saturday after I rebooted my Synology NAS. After I updated my Synology 209 to the latest DSM version, the first thing I noticed was that the NAS sent me emails that Hard disk 2 was having problems; I logged in and saw that it had crashed!

Volume 2 has crashedOh no, not now! I have so many things to do, I can’t handle this right now…

I had no idea how bad the condition of the HDD was (what does crashed really mean?), so I searched through some log files for additional information and found out that the hard disk was suffering from read errors. Okay… lets see how much of the data can still be saved, cause I didn’t like the idea of losing everything I did that day – I just hate doing things twice.

Fortunately the HDD could still be mounted in read-only mode, so I started searching for some free disk space (about 1 TB needed) and found some on my PC. A hard disk on which I store parts of my code still had ~500 GB of free space, so I copied parts of the crashed volume there. And my other NAS, a Synology 109 which I mainly use as “backup of the backup” (paranoia? I call it experience ;-)) and which is normally located off-site 1km away from here, was accidentally here in our house for doing some updates & maintenance. I still had ~400 GB of free disk space there too.

So I scheduled some backup jobs and let them run; this evening, almost 48 hours later, they were all¬†successfully¬†completed, phew! ¬†And from what I can see, nothing has been lost. I’ve ordered a new HDD (2 TB) ¬†which will arrive tomorrow. After it has been installed I’ll have to wait until all the data is copied back to where it belongs; so with a bit of luck everything is up and running again on Thursday.

That’s 5 days of doing almost nothing else but checking the progress of backup jobs… what a waste of time. Well, that’s the negative approach – in fact, I’m lucky that I was still able to access the crashed disk, make a backup of the data and not lose a single byte! Another comforting thought was that no matter how bad the crash would have been, I would still have a 2nd daily backup of all the really important stuff…

Just think of it, the data on all your storage devices does not only represent a lot of bytes, it actually represents a part of your life – photos, video, code and all the other personal data you’ve collected through the years. And they’re worth a good backup plan!