Home Automation with Node.JS & MQTT

Shutting down my old homebrew Windows-based Home Automation system and letting the new Node.JS/MQTT based HA system take full control last Saturday was done without a glitch! Better, faster, smoother than I expected.

I had planned to start with this on Saturday morning, so that I would have ~36 hours to fix any problems that would arise, but that didn’t go as planned – some other things I had to do on that Saturday made me postpone the big switch to Saturday evening.

Around 7 o’clock in the evening I told the rest of the family that they just had to pretend that I was away, unreachable, until I would tell them that I was back again – the ‘do not disturb’ door hanger 😉

The main concern in this whole exercise was not losing any historical data. First I did a test-run of copying all the historical data from MS SQL to MySQL and checked if this still worked like it should; it did. Checked the information in the MySQL database for consistency, correctness and so forth. Great. I ran the historical-data copy again, renamed some tables in MySQL, changed some configuration settings (database names), restarted some Node services and checked if storing the historical data worked as before, but now by Node.JS, directly into the production database. This was the point where I had to decide for the go/no-go; within just a couple of minutes I knew it was a GO!

I copied a file with the new configuration settings to the Raspberry Pi’s and after that, all I had to do was restart all the services so that they would switch from the MQTT broker running on the Windows VM to the broker on the Cubietruck. Great!

About an hour later, after testing some things by walking through the house, pushing some buttons, seeing lights being switched on by detected motion and other stuff like that, I knew I would have a huge amount of free time to do a lot of other things again! 🙂

The day after went just as smooth and relaxed – no issues, just a minor thing I forgot about and which was fixed in 5 minutes… Now, Wednesday evening and 96 hours later, everything’s still 110% OK.

As of now, my Node.JS/MQTT based HA system has the following ‘features‘:

  • Smart meter

The output of the so-called P1 port of our Smart meter is being parsed by a small script which publishes the relevant information (power usage, gas usage) to the MQTT broker.

Our Roomba 563 robot vacuum cleaner is monitored and controlled with a Thinking Cleaner module which is plugged into the SCI port of the Roomba.

  • RFXCOM receiver

Two RFCOM receivers are used to collect information from various sensors (mainly Oregon Scientific temp/humidity, Visonic door/window, motion).

This is one of the drivers that’s controlling a lot of lights in our house, but is also used for things like controlling the garage door opener. I use 2 PLCBUS controllers – one in the meter cabinet, the other one is located at the other end of the house.

  • EISCP protocol

The RS-232 port on our Onkyo AV Receiver enables us to control every aspect of the device – switching HDMI inputs, volume level, on/off, mute…

  • NMA

Notify My Android is being used to send notifications to my cell phone – stuff like new LED Bar messages, and warnings about things that might need my attention.

  • Mobotix

Our Mobotix D22 security camera has a great light sensor; i use this light sensor to determine when the time has come to switch some outdoor lights (front door, back door, garden, gazebo) on or off. The http interface of the camera enables me to take snapshots.

  • LED Bar

Just a funny gadget..

  • IRTrans

The IRTrans LAN module is used to control our UPC media box and to turn on our Dune media player.

The Dune IP Control protocol enables the system to control our Dune HD Max media player.

The HA7Net provides information about in- and outgoing temperatures of our 5 floor heating groups with 1-Wire sensors and about the amount of DHW (hot water) we use (water meter with pulse + Dallas 1-Wire DS2423 counter).

  • Remeha Calenta

One of the most informative devices… even things like the fan speed can be monitored with its ‘service‘ port! More interesting of course are things like modulation level, water pressure, operating mode (central heating, DHW).

  • Alphatronics

A great receiver to receive Visonic Keyfobs and sensor RSSI information;

Primarily used for the Philips Pronto TSU9600, our single remote solution for all our AV equipment.

  • Somfy RTS

An RS-485 Somfy RTS transmitter enables us to control 12 roller shutters.

  • Rules engine

The part of the system that does the real automation: based on inputs (sensors) this engine can initiate all kinds of actions with all the hardware (actors) which is connected to the system.

  • RGB LED

A DMX based RGB LED driver controls 6 RGB LED lights under our gazebo;

This great device enables us to set the room temperature to what we want it to be, without having to walk to the room thermostat or even being at home.

  • Nemef Radaris Evolution

This RFID lock that’s on our frontdoor, controlled by a Nemef RF controller, gives detailed status information about the RFID tags being used, access control and remote access.

  • Conrad MS-35 LED driver

A couple of these are used to control warm white LED strips. I made them wireless with small TTL-to-Wifi adapters.

  • Siemens M20T GSM modem

This GSM modem is being used to send SMS messages to my cell phone, but it’s task is gradually taken over by NMA.

  • Email

Emails are primarily used to notify me about sensors that need new batteries.

  • ELV MAX!

The ELV MAX! Radiator Thermostats are used to control the temperatures in all the rooms in our house: bedrooms, bathroom, etcetera.

Why? Because I can! 😉

  • Zigbee

I have several sensors based on a combination of JeeNodes with XBee ZB modules: motion, temperature, pressure. I also use those XBees to make 3 Chromoflex LED drivers wireless by connecting an XBee to the serial port of the chromoflex – works great.

  • 16-channel LED driver for staircase lighting

Homebrew LED driver to control 13 (or is it 12?) LED strips which light the stairs. The Node driver mainly controls how an Arduino sketch should behave.

  • RFXCOM transmitter

This RF transmitter has only one purpose: controlling 2 433 MHz door bell chimes.

  • Plugwise

12 Plugwise Circles are used for monitoring power usage and to detect whether the washing machine or dryer has finished its program.

  • Chromoflex

This service calculates the payload that has to be sent to the Chromoflex LED drivers to control the LED strips. This payload is then forwarded to an XBee radio to which the Chromoflex driver is connected.

  • Btraced

Btraced is an app for iPhone/Andriod and it enables you to send your location to your own server; this service adds some additional information (by reverse geocoding) so that the location can be displayed on our touchscreen with use of Google Maps.

  • Visonic PowerMax Plus

Our security system is connected too; we’re no longer limited to using keyfobs, panel or keypad anymore for controlling this security system. And additionally, all the sensor information (open, closed, motion, battery status, tampering) is available in my system.

In use since I ditched A-10/X-10 recently. It has been on the shelf for some time after doing some small tests with it, but now it’s an excellent replacement for the small amount of X-10 stuff I was still using.

  • Doorbell

Our Ethernet-enabled doorbell communicates with a Node script to report ‘rings’ and query daylight status (used for switching a LED on & off for visibility of the button).

Quite a list, if I may say so … and it’s all running great!

My ASP.Net website still uses a MS SQL Server for its data – this database is now kept up to date by a NodeJS script, just like the MySQL database on the Cubietruck.

So, what’s next? Well, first I’m gonna take a break, that’s for sure… I’ve done enough JavaScript in the past 8 months, so it’s time for something different; I’ve also neglected some other things that really do need my attention now. And of course I’ll have to start working on the User Interface, for which I’l have to learn a lot before I can start developing.. never a dull moment!

Onwards!

Some last changes before Node.JS takes full control

X-10 LM15The actor in the picture to the left was probably one of the first real Home Automation related pieces of hardware I bought. Everything I did before I had this LM15 installed was only about monitoring. It must have been sometime in 2007. This LM15 device, used for the outside light at the front door, was controlled by a ACT TI-213 X-10/A-10 controller. But now, 7 years later, those items have to go. Not because they’re broken, but because the A-10 signals put on the power-line by the controller mentioned above don’t always reach 2 other A-10 modules anymore (which are located at the other end of our house). Result: garden lights being on all day. So it’s time to say goodbye to my A-10 controller and 3 A-10 actors. I didn’t want to use a technology that only works 70-80 % of the time anymore – so why spend time on writing a new driver for it; it will be replaced with PLCBUS modules and Insteon LED bulbs.

Simplecortex

Another thing I did during the last weekend was replacing 2 Simplecortex boards. One was used for my Opentherm Gateway and the other one for the smart meter. I changed my mind about the MQTT topics these boards had to publish their information to, so to change that I had to install the CooCox CoIDE again, install libraries, change the code, build, flash, … – too much work!

So I decided to connect the Opentherm Gateway to a Raspberry Pi (RPi) using an old FTDI breakout board  and do the same with the smart meter using a USB-cable with a RX-inverted FTDI chip inside. Now all I had to do was write 2 small Node.JS scripts – which I can change on the fly in every aspect with just a simple text editor – notepad will do just fine, although using a more advanced editor like NotePad++ or UltraEdit would be a better choice. Long live convenience!

So why am I doing all this? It’s all for the upcoming “big shutdown” of my old Home Automation system, of which the next-gen version is eagerly waiting to take over:

HA system overview

The image above shows the status of what has become of what I started in June last year.

40 services in total, running on 4 RPi’s and a Cubietruck. Based on about 15,000 lines of Javascript code. A part of those services (processes) just share their information (sensor drivers like for RFXCOM, HA7Net), others also accept commands (PLCBUS, AV equipment, ELV MAX!, Zigbee) and a few others are just facility services like taking care of historical data, metadata etcetera.

All this should be able to run on a single Raspberry Pi regarding CPU usage as you can see, but that won’t work in my case, because all those separate Node.JS processes use quite a lot of memory, so I had to use a few more RPi’s to get them all up and running.

Right now, both HA systems are operational – one a bit more than the other though; almost all tasks in my Windows system have been deactivated and all that’s left of it is that it’s still storing historical data in a MS-SQL database. The other one (the Node.JS based one) does it all and even more, cause it’s also feeding the MS-SQL database with historical data, but puts it into a ‘shadow’ database for now.

So now the time has come to click that close button in the upper right corner…
close

What will happen, I don’t know. Hopefully nothing serious…

Immediately after I’ve shut down my old system, I’ll have to restart a script on one of my RPi’s so that it starts storing the historical data in the ‘live’ MS SQL database instead of the shadow version – my website needs this MS SQL database because it uses it to create charts and read the current device statuses. This will stay that way until I’ve developed a new website.

What more is there to do? Nothing I can think of right now, actually… However, I still find shutting the old system down a bit scary; I don’t want to find out after a few days that I forgot something!

So what I will do is plan this action for the next weekend, on Saturday, early in the morning. That will give me time to monitor everything and, more important, immediately respond if something does go wrong… exciting!