Happy new year!

 

I’m ready for it as you can see, are you? 馃檪

Ready for New Year's Eve

 

Let’s hope 2015 will bring just as much Home Automation fun as 2014 or even more! Some exciting Kickstarter projects will probably arrive and I’ve got some plans of my own, and they’re all fighting for my attention 馃槈 As usual, you’ll read all about it over here. Happy New Year!

RaZberry: upgrading to 2.0.0 and modules

RaZberry on a PiI’m beginning to like the RaZberry, despite the problems I had with it the last few days. Why? Cause it’s open, you can add your own stuff to it- it’s different from all those other closed systems you see so often.聽Should have done more with it much sooner … I bought the RaZberry @ CeBIT 2013, ordered a Wall Plug when I got home and tinkered with both for a while when that Wall Plug arrived. But since I had no plans on buying more Z-wave hardware (it wouldn’t add anything I didn’t already have – well, problemz maybe), the RaZberry board soon ended up on a shelf cause I needed the Raspberry for something else.

Until I got my hands on a Fibaro motion sensor 2 weeks ago … this sensor was almost ‘irresistible’, I just had to use it. So that’s when I picked up the RaZberry project again to see if I could get the RazBerry integrated into my own HA system.

After I made some progress, I mailed Pieter聽(who also bought a RaZberry at the CeBIT and ordered the same motion sensor) and he installed my code. But he encountered strange things; weird behavior and messages like “Cannot set port to 8083: Cannot bind to port”, “Object.keys called on non-object”, needing to reboot in order to get things going again etcetera. Strange …

Browsing the web I found out that there was a new 2.0.0 firmware for the RaZberry聽so I decided to upgrade. Suddenly I had exactly the same problems as Pieter; so I reverted back to 1.7.2 and the problems disappeared again.. So Pieter must be running on firmware v2.0.0 – we checked this and the source of the problems was found. Something must have changed that resulted in my code crippling the whole RaZberry environment…? So I kept my RaZberry running on v1.7.2 until I had the time to find a solution for this, cause staying on v1.7.2 just wasn’t a solution.

This afternoon I had some time to resolve the issue. Although being a complete newbie on Z-Wave (and RaZberry in particular), I did realize that editing the main.js file to get my own code running was not such a good idea; an upgrade made the changes to main.js disappear so there would probably be a better way to extend the RaZberry system with my own add-ons. User Modules seemed like the way to go, as if it was made for what I was doing 馃槈

So I put my code, almost unchanged, into a User module. It still didn’t work… same problems. Binding worked and the callback was still being executed. Although not completely, so something in my code was still blocking something … I figured out that it must be the blocking /ZWaveAPI/Data/… HTTP call I did, so I rewrote that one to an async version and my problems were gone. Why? I still don’t know; for that I don’t know enough (about how the RaZberry API’s work internally – but I’m glad I found a way to get my upload to my HA system working again :-).

After a while I found out that my Usermodule didn’t always start – it felt like some dependency wasn’t met after a restart (sudo service z-way-server restart) so I changed the module.json and added a聽autoloadPriority and set it to 9999, like this:

{
  "autoload": true,
  "singleton": false,
  "autoloadPriority":9999,
  "defaults": {
    "title": "HA system binding",
    "description": "Report sensor values to my HA system by HTTP",
    "device": ""
  }
}

Now my module will be loaded later then all the rest; found it on the聽Z-Wave-Me/home-automation Wiki on Github. The code is still very fresh, it’ll need some more time to prove itself, but so far everything is looking good.

BTW, how nice would it be if MQTT would become part of the RaZberry system …

The Fibaro FGMS-001 on RaZberry and Z-Wave, The Interoperable Standard ?

Well, maybe the fact that the Z-Wave logo can only be found at the bottom of the package聽of the FGMS-001 was an omen for another story about bad luck. Here it goes.

Fibaro FGMS-001After successfully binding to events in the Z-Way API and getting the required information into my home-brew Home Automation system, I focused on the FGMS-001.

It’s a real beauty; small, flexible, you could almost say that Fibaro reinvented the motion sensor and even managed to fit some additional sensors into the ‘ball’ which are useful @ lots of places in the house.

So I was very excited to see this sensor working. Inclusion went fine, but then the trouble started. My binding worked fine, but no sign of Temp & Lux values? I also saw an error in the z-way-server.log:

Packet CC::MULTI_CHANNEL_CMD_ENCAP_V2 is too short: required at least 5 bytes, got 3
Error returned from _zway_cc_call_handler(zway, command, controller->id, 0, data[4], &data[5]): Wrong packet from Z-Wave network or Discovery got bad data (stick communication failed) (-9)

That doesn’t look good, so I searched for the error message; according to this seems I need a better/newer XML file, so I did. Motion was working, but Temp & Lux weren’t being updated. After a while I found out that this doesn’t happen by design聽– you have to change some parameters in the configuration. OK, did that. Wrong – you can’t use the RaZberry Expert UI to change those values, cause it messes up the entered values – enter “10” but this is changed to “32769” once you tab your way to the next textbox. Imagine what happens after applying such a value – temperature measurement every 9 hours.

So to finally get things going, I had to do it myself from a command prompt (setting Temperature interval to 600 seconds):

wget http://192.168.10.xxx:8083/ZWaveAPI/Run/devices[7].instances[0].commandClasses[112].Set(64,600,2)

Yep, by the time I got this far the device counter had already reached 7, with just 2 sensors. And the battery is at 88% now, after almost 3 days – what was it, 2 years battery life? I’ll believe it when I see it. The FGMS-001 has lost a lot of its ‘glamour’ in just 2 days.

So what is this, a sensor from a bad batch? Still in beta stage? Z-wave.me forum blames Fibaro for not following the standards of the holy grail of Home Automation called Z-Wave. Is this true and if so, why do this? Can’t they read? 聽Or is there some other purpose for this kind of deviation? I don’t know and frankly it doesn’t even really matter – what it results in does matter.

Cause all these little deviations make Z-Wave unreliable, immature – you never know whether it’ll work out of the box or not. Lots of techies will be able to solve this sort of issues, but the majority won’t. Killing. And imagine the IoT, with聽a predicted >25 billion devices around 2020, being driven by Z-Wave … it gives me the shivers!

Razberry, binding to events and communicating device state changes

It all started about a week ago, when I became the owner of a very good looking Fibaro Motion sensor:

Fibaro FGMS-001

Compared to a Visonic motion sensor (to the right), this Fibaro is small. And very nice to see as well if I may say so. And that’s not it, there’s more inside than just motion:

  • A temperature sensor;
  • Light intensity;
  • an accelerometer to detect tampering.

And now that I’ve got one, it was time to give the Razberry, which has been unused for the last 18 months, another try.

So I took a spare Raspberry Pi, downloaded a complete image from here聽and started fiddling with the web-interface which is at http://<RPi-IP-address>:8083 . I have more Zwave devices than just this Fibaro FGMS-001 Motion sensor, so I chose to start with including a Fibaro Door/Window sensor, the FGK-101 – because it’s easier to test with a sensor on your desk that doesn’t require you to wave at it all the time 馃槈 All went well, I saw the sensor switching from on to off and vice versa.

Now how do I get this infomation into my own Home Automation system without too much hassle? I’m used to subscribing to MQTT topics lately, so I started searching for a way to catch events or hook up to ‘something’ that could help me to export device data in some way. The Developers Documentation PDF聽gave a clue: binding. Ah, so that’s how it’s called in Z-Way world.

The PDF says: “A very powerful function of the JS API is the ability to bind functions to certain values of聽the device tree. They get then executed when the value changes.” Yeah, that sounds like what I want. Lets give it a try.

To run the “binding to *everything*” code I had to make sure an additional (Java-)script was executed when the z-way-server service was started; for that I added an additional line to the end of the file聽/opt/z-way-server/automation/main.js file (showing just the last 3 lines here):

  });
  executeFile('bind_all.js')
}

And this bind_all.js file, which has to be placed in the same directory as the main.js we just changed, goes like this:

var callback = function(type,arg) {
  console.log('### here I am ###');
};

for (var dv in zway.devices) {
 var dv = zway.devices[dv];
 dv.data.bind(callback, null, true);
}

I restarted the z-way-server service (sudo service z-way-server restart) and moved the magnet of the Door/Window sensor away; bingo! The ‘### here I am ###’ showed up in the z-way-server log (tail -f /var/log/z-way-server.log). OK, nice.

Now what? I had a look at what others did; the only thing聽聽I could find was executing a system() call to notify the outside world that something had changed but I didn’t like that approach; sounds like too much overhead? Getting a MQTT client running would be nice, but way out of reach with what the JS API has to offer 馃槈

I preferred to keep it all wrapped up in a single script without any other dependencies, so I went back to the聽Developers Documentation PDF I mentioned earlier. It says: “The JavaScript implementation of Z-Way allows to directly accessing HTTP objects”. So I can do a HTTP request from inside my callback? Let’s try it. With a few lines of code I managed to POST the result of a JSON.stringify(this) to a NodeJS HTTP server that was running on my PC. Nice, and the information I was looking for was in there, great. But this POST resulted in 15 KB of data (including headers and other usual stuff), just for an opening/closing binary sensor? Come on, still not good enough!

Now the JSON API came to the rescue; again from the Developer Documentation PDF: “/ZWaveAPI/Data/<timestamp> returns an associative array of changes in Z-Way data tree since <timestamp>”

Let’s use that and POST that response to my little HTTP server. Yes! Same result (as in: it also contains the information I need) and the size has decreased to 1/9th of what it was.

About 1700 bytes to ‘communicate’ a state change of a binary sensor is still too much, but I couldn’t find any other way to do this. Maybe I’ll parse the result further inside the callback to minimize the I/O to a more ‘sane’ amount. So here’s the complete bind_all.js script that’s now communicating state changes of my Fibaro Door/Window sensor:

var callback = function(type,arg) {
 try {
   var options = {};
   // timestamp
   var ts = (new Date()).getTime();
   ts = Math.floor(ts/1000).toString();
   // query local ZWave API
   options.url = "http://127.0.0.1:8083/ZWaveAPI/Data/"+ts;
   options.method = "GET";
   var res = http.request(options);
   // send response to HA system
   options.url = "http://192.168.10.xxx:8181";
   options.method = "POST";
   options.data = JSON.stringify(res);
   res = http.request(options);
 } catch(err) {
   debugPrint("Failed to upload device state changes to HA system: " + err);
 }
};

for (var dv in zway.devices) {
 var dv = zway.devices[dv];
 dv.data.bind(callback, null, true);
}

Now back to the sensor which started all of this – the Fibaro FGMS-001.

In a 2nd post, because it’s quite a story … soon!

Solar Panels and the Omnik Wifi Kit

As announced, last Thursday we went solar!

Men at workIt took me some time to take that final step and order the solar system, cause I’ve been interested in solar energy for a long time. But now that we finally have 12 solar panels on our roof I’m very satisfied with the end result.

I decided not to do this myself cause I knew that it would take me much too long – much, much longer than the 2 professionals did. They were done in about 6 hours and left me with a fully functional solar system:

Almost finished

4 rows of 3 x聽JA Solar JAM6-60/280-PR panels and an Omniksol-3k-TL inverter in the garage:

Omink inverterI’m very satisfied with how it all went – from the first contact with Sungevity to the point where the solar system was working.

sungevity

 

 

 

No loose ends, everything is taken care of for you, from notifying 3rd parties that I’m gonna start producing energy soon and even the VAT refund (the EU considers you as a energy producer) is handled 聽for a small fee.

All I had to do was sign some papers and provide the password of my Wifi Access Point to connect the Omnik inverter to Internet. Great job guys (and gals)!

Now on to the tech part of this post: the Omnik Internal Data Collector and how to use it.

Now that we have 12 solar panels on our roof I’d really like to be able to monitor them – myself, local and not on some portal.聽Fortunately Sungevity made it a standard procedure to install the Omnik Wifi kit. This kit uploads the performance data to the cloud and can be viewed from the Omnik portal.

Wifi Kit inside the inverterBut it would be even better if I’d be able to receive this information directly from the Omnik Wifi kit instead of heaving to log in on some portal and probably do some web scraping to get the data.

You can see the Wifi antenna of the kit on the picture to the left, right next to the grey cable. Now the really great part of this Wifi kit is that you can add your own ‘remote servers’ from the web interface – for instance, you can add your PC (bad choice) or a Raspberry Pi (better), or whatever – just enter the right IP address and port number and you’ll receive the same information that you’ll see on the Omnik portal. (you’ll have to do the charting thing yourself though 馃槈

The information looks like this (this one is invalid for obvious reasons):

68 81 41 b0 77 6f 7e 5f 77 6f 7e 5f 81 02 01 4e  h?A掳wo~_wo~_?..N
4c 44 4e 33 30 32 30 31 34 35 53 32 33 35 35 00  LDN3020145S2355.
45 08 42 08 3d ff ff 00 00 00 00 ff ff 00 00 ff  E.B.=每每....每每..每
ff ff ff 08 89 ff ff ff ff 13 8b 00 00 ff ff ff  每每每.?每每每每.?..每每每
ff ff ff ff ff 00 00 00 00 00 31 00 00 00 17 00  每每每每每.....1.....
00 00 00 00 00 ff ff 00 00 00 00 00 00 00 00 00  .....每每.........
00 00 00 00 01 4e 4c 31 2d 56 31 2e 30 2d 30 30  .....NL1-V1.0-00
38 33 2d 34 00 00 00 00 00 56 32 2e 30 2d 30 30  83-4.....V2.0-00
32 38 00 00 00 00 00 00 00 00 00 00 00 6d 16     28...........}.

Having the luxury of knowing what the Omnik Wifi kit uploads to the cloud and being able to see what it results in on the Omnikportal should not make it too hard to figure out the ‘where is what’.

But hey, wait, I’m not the first one that can think of this – let’s do some serious searching – maybe someone already figured it all out?聽And yes, there are varioussources, even with code. With that information I had my NodeJS script running in no-time聽and saw the information being received. Nice, cause now I have the same information as can be seen on the Omnik portal. But I’m not done yet, cause I wanted some additional checks on the data received from the Omnik Wifi kit.

From the source code made by others I knew that the ‘checksum’ used was just a sum of all the individual bytes in the packet.聽So it wasn’t so hard to find out that the packets being received used the same method;聽in the example packet above, the 0x6d at the end is the low byte of the sum of all preceding bytes except the first one (the start-byte, 0x68).

Another thing is that it looks like the packet has a fixed header and always ends with the checksum + stop-byte (0x16). The size of this all is 14 bytes. So with a packet of 143 bytes this leaves 143-14 = 129 (0x81) bytes for the payload – aha, the 2nd byte in the packet is the payload size! Some more validity checks 馃槈

Right now my NodeJS script is running on a Cubieboard, collecting data and checking if the additional checks really work.

Have fun!