Migrating to the future has begun

I think I’ve got it. For now… Almost a year ago I realized that something had to change; my Domotica system grew too fast, became too big to keep it all in a single executable: stability and flexibility were the two main issues that had to be addressed.

During the last 12 months I tried several things to find a better solution on how to gradually rebuild my system; ZeroMQ, SimpleCortex, MQTT, FEZ, Raspberry Pi, Node.JS, Python, Netduino, they were all tried and tested for some time. And (for me) the winners are: Raspberry Pi, MQTT and Node.JS.

The power of Node.JS enables me to very quickly develop a stable hardware driver and combining this with a very small, low power & cheap Linux box like the Raspberry Pi to run those drivers is absolutely unique in my opinion; and MQTT is the super-glue that makes all the different parts of the system talk to each other, from Arduino sketch to VB.Net application.

The last weeks have been quite busy for me, there was very little time was left for working on Domotica, but with some hours here and there I still managed to write 5 drivers for some parts of the hardware I’m using in my Domotica system. So since a week or two I have a Raspberry Pi here that has been running those 5 replacement-drivers flawlessly – for my RooWifi (Roomba), RFXCOM RF receiver, Mobotix security camera light sensor, HA7Net with 10 1-Wire sensors attached to it and for my Remeha Calenta boiler. The last one mentioned is one of the most CPU & I/O intensive drivers I have, but the Node-version of all those drivers work perfectly on a single RPi:

uptime

Still enough processing power left to add some more drivers, don’t you think?

I made some changes to my monolithic Domotica system so that it would accept ‘raw’ device information from the outside world by means of MQTT, automated starting the drivers after booting the RPi and everything has been running great so far. I still have a lot of things to do though, mostly regarding maintenance & ease of use, of which some issues have already been addressed and others need some more time to find the best solution:

  • backup procedure for the RPi;
  • remotely controlling the RPi and its drivers;
  • supervising the RPi and its drivers;
  • storing global configuration data elsewhere.

So I still have some things to do before I can concentrate on migrating all my hardware drivers to the RPi, but I think I should be able to do one driver per week (that includes developing, testing and accepting it as a reliable replacement). The advantage I have is that I already have thoroughly tested Delphi code with all the workarounds for hardware specific peculiarities in the code; for example, I know that the Remeha Calenta sometimes doesn’t respond to a query, so I already know the new driver on the RPi needs to able to handle that – knowing all those peculiarities will certainly speed up things.

Another advantage is that all my hardware interfaces are connected to my LAN, so I don’t have to worry about RS-232, -485 or USB stuff, an IP address is enough for the RPi to communicate with the hardware interface.

So if all goes well, my Domotica system will be stripped of all its built-in drivers in about 30 weeks or so (cause that’s about the number of drivers I’m using right now) and all those drivers will be migrated to the RPi. Sounds doable, and I hope this will still leave some time to do other things as well, like adding more hardware, software and features to my system.

Yeah I know, I’m always too optimistic… 😉

RooWifi, Node.js and MQTT working together

As I wrote yesterday, the RooWifi has both a Web Interface and it can also connect to a remote TCP server. The RooWifi prioritizes the TCP server connection to the Web Interface, so if there’s a remote TCP server running and accepting connections, this TCP Server is in full control. The IP address and port number can be set from the RooWifi Web Interface:

RooWifi TCP server settings

This afternoon I tried to get the RooWifi to connect to a TCP server. I decided to use Node.js this time. I’ve used Node.js before and I wanted to see if I could get it running on one my Raspberry Pi‘s, create a TCP server, accept connections, parse a JSON payload and send the results to a MQTT broker (mosquitto).

I remembered a set of commands on JeeLabs to install Node.js, so I used that:

sudo usermod -aG staff pi && sudo reboot
v=v0.10.7
cd
sudo curl http://nodejs.org/dist/$v/node-$v-linux-arm-pi.tar.gz | tar xz
cp -a node-$v-linux-arm-pi/{bin,lib} /usr/local/

The first script I made was a TCP server that kept track of the clients that connected and disconnected and wrote the incoming data to the console:

net = require('net');
var connections = [];

net.createServer(function (socket) {

socket.name = socket.remoteAddress + ":" + socket.remotePort
 connections.push(socket);

socket.on('data', function (data) {
 var newDate = new Date();
 var time = newDate.toLocaleTimeString();
 console.log(time + " | " + socket.name + " < " + data);
 var obj = eval('('+data+')');
 console.log(time + " | temp=" + obj.roomba.temp);
 console.log(time + " | dirt=" + obj.roomba.dirt);
 });

socket.on('end', function () {
 var newDate = new Date();
 var time = newDate.toLocaleTimeString();
 console.log(time + " | " + socket.name + " ended the connection");
 connections.splice(connections.indexOf(socket), 1);
 });
}).listen(8001);

console.log("Server listening on port 8001n");

That’s it?? Yep.. 26 lines of code, wow. The output looked like this:

19:57:29 | 192.168.10.201:1561 < { "roomba": { "status": "3", "cleaning": "0", "battery": ":0", "temp": "36", "dirt": "0" } }
19:57:29 | temp=36
19:57:29 | dirt=0
19:57:34 | 192.168.10.201:1561 ended the connection

I also installed the MQTT package from adamvr:

npm install mqtt

I added a mqtt client to the script so I could publish all the information inside the JSON data and made some preparations to receive commands from the outside world as well:

net = require('net');
var mqtt = require('mqtt');
var connections = [];

var mqttc = mqtt.createClient(1883, '192.168.10.17', {
 keepalive: 30000
 });

mqttc.on('connect', function() {
 mqttc.subscribe('command/roomba');
 mqttc.on('message', function(topic, message) {
 console.log('topic: ' + topic + ' payload: ' + message);
 });
});

net.createServer(function (socket) {
 socket.name = socket.remoteAddress + ":" + socket.remotePort
 connections.push(socket);

socket.on('data', function (data) {
 var newDate = new Date();
 var time = newDate.toLocaleTimeString();
 console.log(time + " | " + socket.name + " < " + data);
 var obj = eval('('+data+')');
 for(var key in obj.roomba){
 console.log(time + " | "+key+" "+obj.roomba[key]);
 mqttc.publish('/value/roomba/'+key, obj.roomba[key]);
 }
 });

socket.on('end', function () {
 var newDate = new Date();
 var time = newDate.toLocaleTimeString();
 console.log(time + " | " + socket.name + " ended the connection");
 connections.splice(connections.indexOf(socket),1);
 });
}).listen(8001);

console.log("Server listening on port 8001n");

Still a very tiny script! I started the script in the background and let it run for a while (it’s still running):

node tcpmqtt.js &

Now let’s see if I can get the RooWifi information visible on another machine, my Win7 PC for instance:

RooWifi data published

Bingo… Now it’s easy to write a webpage to display the information, make certain values historic in my SQL database, control the Roomba from any User Interface (Touchscreen, Smart phones, tablets) – whatever you can think of!

Of course, I still have to add the code to control my Roomba, but that’s just a matter of time and will probably add just a couple of lines of code to the script.

What I’ve learned today is that the Remote TCP server feature of the RooWifi is the best way to monitor & control your Roomba and that Node.js is very powerful and most important: it works like I think: event-driven.

Time-lapse Video with the Raspberry Pi

Last Thursday I taped the Raspberry Pi Camera Board to the glass, pointed it towards the sky and just let it run for 24 hours… the result is at the bottom of this post.

Camera taped to the window

I updated the RPi to the latest firmware, because I read that this would bring back the feature of turning off the camera LED with the ‘disable_camera_led=1′ setting in config.txt. And since I had never done a firmware update before, I thought this was a good reason to give it a try.

sudo apt-get update && sudo apt-get -y dist-upgrade
sudo apt-get install rpi-update
sudo rpi-update
sudo reboot

There’s another workaround for disabling the LED though, which is a small Python script. I mounted a NAS share on my RPi so that I didn’t have to worry about the size of all the stills that would be generated by this time-lapse adventure, for that I had to edit /etc/fstab and add the following line:

192.168.10.1:/volume1/exch /mnt/exch nfs nouser,atime,auto,rw,dev,exec,suid 0 0

… and of course I had to add the appropriate privileges on my NAS for the RPi.

On Wednesday around 23:00 I started the time-lapse run:

cd /mnt/exch/stills
raspistill  -o still%06d.jpg -t 999999999 -tl 10000 &

This will start raspistill and put it in the background. I closed the connection to the RPi, shut down my PC and went to bed. The next morning I checked the ‘stills’ directory on my NAS from my Android phone with ES File Explorer and saw a couple of thousand files – it’s still working, great 😉

Thursday evening, after 10 o’clock or so, I killed the raspistill process and I had about 8000 stills. But unfortunately no time to do something with them, for that I had to wait until Friday evening.

For creating the time-lapse video from the stills I used Sony Vegas Movie Studio HD and opened the huge set of stills this way (Project > Import Media):

Opening the stills

This way you’ll see all the stills as 1 single ‘file’ in the Project Media. I dragged the media to the Video track, set the rendering quality I wanted to use and about 8 minutes later the video was ready, 547 MB in size. Uploading to Youtube took a bit longer though… about 5 hours…

It was very cloudy that day, so the video is not really that entertaining.. but it was fun to do; let’s hope the weather this summer will give me the opportunity to make some nicer videos than this one… 😉