Smart meter is on its way

Smart meterLiander, our grid operator for electricity and gas, started installing Smart meters in January 2012. Liander expects that in 2020, 80% of all the households in the area they’re responsible for, will have a Smart meter.¬†I don’t like waiting that long and that’s why I applied for a priority installation of a Smart meter in our house.
That was last weekend; last Tuesday we got the confirmation and next Monday the Smart meter will be installed! ūüôā
Wow, that’s fast; probably because the priority installation comes with some extra costs – namely 70 Euro.

But … this is a bit too fast, actually..¬†I mean, once the new Smart Meter has been installed, we won’t have a gas meter with a reflective “6” anymore which means I won’t be able to monitor our gas usage anymore. Something needs to be done about that, and fast; I hope our kids won’t read this, cause they’ll take advantage of this and immediately turn the shower knobs to ‘full steam’, knowing that I can’t keep an eye on their gas usage! ūüėČ And I won’t be able to spend time on this before Wednesday, or even later; but there’s hope…

The Smart meter that will arrive next Monday will have a so-called P1 interface. Maarten Damen has already experimented¬†with this and also developed a SmartMeter Plugin for his HouseAgent; this looks like a good opportunity to broaden the horizon and look beyond the software I make myself and see what’s available ‘next door’, as¬†I mentioned¬†earlier.

Well, I hope I can say more about how this worked out soon!

Receiving sensor data with ZeroMQ

Mobotix cameraTo see what has to be done to get ZeroMQ working in my Domotica system, I decided to do a small test. For this test I chose a sensor that’s inside one of my camera’s, the Mobotix camera you see in the picture to the left. This Mobotix has a very good illumination sensor which I use for detecting daylight. The Mobotix can be configured to send (among a lot of other things) the sensor value at a regular interval to a list of IP addresses. My camera is configured to send it’s IP address, a PIR value and the illumination value each minute,¬†separated¬†by a semicolon. This results in the following data, with a 1 minute interval between each line:

192.168.120.11;0;25;
192.168.120.11;0;30;
192.168.120.11;0;35;
192.168.120.11;0;46;
192.168.120.11;0;52;
192.168.120.11;0;63;
192.168.120.11;0;68;

Let’s see if I can get this into my system by sending it to a ZeroMQ server that’s embedded in my system; so for that I need 2 things:

  • A 0MQ client, written in whatever language I want (Delphi XE2¬†this time);
  • A 0MQ server in my Domotica system, written in Delphi 2005.

I started with the client, following the examples found in the ZeroMQ Guide:

program MobotixNotifyPlugin;
{$APPTYPE CONSOLE}
{$R *.res}

uses
  System.SysUtils,
  IdTCPServer,
  IdIOHandler,
  IdContext,
  zmq in 'zmq.pas',
  zhelper in 'zhelper.pas';

type
  // dummy class for holding the event
  TProcObject = Class(TObject)
    procedure ServerExecute(AContext: TIdContext);
  end;

var
  PO       : TProcObject;
  Context  : Pointer;
  Requestor: Pointer;
  Msg      : string;
  Server   : TIDTCPServer;

function Version:string;
var
  major, minor, patch:Integer;
begin
  zmq_version(major, minor, patch);
  Result := Format('%d.%d.%d', [major, minor, patch]);
end;

procedure TProcObject.ServerExecute(AContext: TIdContext);
var
  IO   : TIdIOHandler;
  data : string;
  reply: String;
begin
  IO := AContext.Connection.IOHandler;
  //IO.CheckForDataOnSource(10);
  if not IO.InputBufferIsEmpty then
  begin
    data := IO.InputBuffer.ExtractToString();
    Writeln(TimeToStr(Now)+' '+data);
    s_sendmore(Requestor, 'MOBOTIX');
    s_send(Requestor, data);

    reply := s_recv(Requestor);
    Writeln(TimeToStr(Now)+' '+reply);
  end;
end;

begin
  PO := TProcObject.Create;
  Writeln('ZeroMQ library version: '+Version);
  Context := zmq_init(1);
  Requestor := zmq_socket(Context, ZMQ_REQ);
  zmq_connect(Requestor, 'tcp://192.168.10.40:5563');

  Server := TidTCPServer.Create;
  Server.DefaultPort := 808;
  Server.MaxConnections := 1;
  Server.OnExecute := PO.ServerExecute;
  Server.Active := True;

  try
    while True do
    begin
      sleep(1);
    end;
  except
    on E: Exception do
    begin
      Writeln(E.ClassName, ': ', E.Message);
      zmq_close(Requestor);
      zmq_term(Context);
      FreeAndNil(PO);
    end;
  end;

end.

Receiving the sensor data from the Mobotix is handled by the Indy TIDTCPServer class and all it does is forwarding the incoming data to my system the 0MQ way, preceded by the Device ID (‘MOBOTIX’) the Mobotix camera has in my Domotica system.¬†That should do it, in a very very simplified way; on to the server part!

procedure TZMQServerThread.Execute;
var
  DeviceID, Data:string;
begin
  while not Terminated do
  begin
    // read Device ID and sensordata
    DeviceID := s_recv(Responder);
    Log('RX '+DeviceID);
    Data := s_recv(Responder);
    Log('RX '+Data);

    // transfer sensor data to Device object
    gInterfaceDataHandler.DistributeToDevice(DeviceID, Data);

    // send reply to 0MQ client
    s_send(Responder, 'OK');

  end;
end;

The code above only shows the code for the Execute() method of a class derived from a Delphi TThread. All this method does is receiving information from the 0MQ socket and ‘injecting’ it into a Interface data queue, which handles the data delivery to the device object. From there, events are triggered, data is made historic, etc. etc. Again, the method code above is very very simple and not good enough for use in a live system, but good enough for testing.

OK, now it’s time to see if this ZeroMQ Plugin (hint, hint) works. I first started up the Plugin and after it had received the first data sample from the Mobotix, I started up a development instance of my Domotica system. And this is the output of both the Plugin and my system:

Plugin:
19:29:02 192.168.120.11;0;629;;
19:29:02 OK
19:30:01 192.168.120.11;0;633;;
19:30:01 OK
19:31:01 192.168.120.11;0;640;;
19:31:01 OK
Domotica system:
19:29:02.419 ZEROMQ : RX MOBOTIX
19:29:02.419 ZEROMQ : RX 192.168.120.11;0;629;;
19:30:01.835 ZEROMQ : RX MOBOTIX
19:30:01.835 ZEROMQ : RX 192.168.120.11;0;633;;
19:31:01.561 ZEROMQ : RX MOBOTIX
19:31:01.561 ZEROMQ : RX 192.168.120.11;0;640;;

Yeeha, it works, I’m receiving data in my system and everything keeps on working!

You might wonder why I’m so thrilled about being able to do something that looks a lot like what I’ve been doing from the beginning (i.e. getting data into my system); well, it is¬†what it looks like, but it’s more!

ZeroMQ enables me to do a lot of things:

  • I can start using any programming language for which there’s a ZeroMQ binding;
  • This programming language¬†independence¬†also gives me platform independence¬†(try running a Delphi-win32 executable on a Linux based NAS..)
  • I can break up my system in lots of small parts, starting with removing all the hardware I/O related code from the ‘core’, resulting in less chance of the whole system collapsing;
  • And with all this, create a multi-language, multi-platform distributed home automation system!

Although this all sounds nice, I’m still far away from implementing all this; this new architecture comes with new questions and demands (and new trade-offs?).

For example, I’ll need to do some sort of authentication for these loosely coupled Plugins, because I don’t want just anything that can ‘do’ ZeroMQ to start talking to my system. And I will also need some sort of heartbeat messaging, so that the ‘core’ knows if all the Plugins are still online and performing like they should. And the Plugins will need to know where to find the ‘core’; and the core should be able to start & stop Plugins, and … ¬†sorry, this post ends here – I’ve got work to do!

 

Ready for the future?

My Domotica systemAlthough my Domotica system is very stable, fast and everything, I’ve become somewhat unhappy with how it has evolved through the years.

It all started in 2006 with monitoring gas usage with a CNY70 and 1-Wire counter (DS2423).

Next came RFXCOM, X10, PLCBUS, IR, and so on. Right now, my system has about 30 interfaces to the outside world, all coded in Delphi; Borland Delphi 2005 to be precise. And all that code results in a monolithic executable of 2.9 MB, uses 1500 KB of memory at run-time with an average of <1% CPU usage on a Intel i3 in a Hyper-V VM.

Nothing wrong with these numbers, right? I could go on for years like this with no problem at all. However, I started having second thoughts about that statement a year ago.

  • With the ever growing number of stuff built into a single executable, the system becomes more and more vulnerable to fatal runtime errors that could make the whole thing crash.
  • I am bound to using Delphi as single programming language for the system. Yes, I’m also using other programming languages like VB.Net, ASP.Net, Arduino, (Pronto-/Java-)script, C# but those are only used for User Interfacing and hardware.
  • Delphi restricts me to Windows as platform for my Domotica system. For example, my NAS could be a good candidate to host some pieces of my Domotica system; I can’t do that right now.
  • It’s hard to make use of other non-Delphi code that can help me to ‘enrich’ my system.

6 years ago I could never have imagined that my Domotica system would grow as fast as it did, can do what it does right now and how important it would become in our current daily lives. And the time will come where the current state of my system becomes a burden with respect to adding new functionality.

Conclusion: I have to become more flexible – break the system up in small pieces and find a way to get rid of the¬†programming language and platform¬†dependency. Cause if I don’t, I foresee problems – either by not being able to keep up with the pace in software development, getting stuck on old operating systems or facing huge investments in writing my own Delphi code for something that’s freely available in some other language.

The big question is: how to do that? I don’t like the idea of porting more than 150000 lines of Delphi code to another programming language, cause that would get me in the same situation as it is now. Nor do I want to stop using the code I wrote during the last 6 years; it works fine, so that would be a complete waste! No, I want to keep using the code I have as much as possible, I should be able to gradually transform my system piece by piece and not be forced to some ‘Big Bang’.

So the first thing I have to do is break the current system up in the building blocks it’s being made of; right from the start I have been working in terms of ‘hardware interfaces’, ‘devices’, ‘sensors’, ‘events’, ‘actors’ in an object-oriented way, so breaking things up into smaller pieces should not be that hard; transforming a interface unit with some classes into an executable should be doable but the biggest problem is that a hardware interface object has to be able to get data from itself to the device object and vice versa: a device object (let’s say an IR controlled TV) also has to be able to send to the IR interface; some internal queues take care of that now, but I can’t use those anymore; those building blocks I mentioned need some way to get data across from one process/executable to another. And I think I’ve got the right solution for that – ZeroMQ¬†(0MQ).

ZeroMQ was first brought to my attention during a meeting last year with some friends who are just as addicted to Domotica as I am and it has been on my mind ever since; but I never took the time to have a really good look at it; since 3 weeks I’ve started reading about it, looking at code samples and got more and more excited and realized that this is just what I need. And I’m not the only one that feels that way, as can be seen here and here.

Last Friday I had a day off and spent most part of the day to get 0MQ up and running in Delphi. I found some¬†Delphi bindings for 0MQ (some more finished than others), tried to let 2 Delphi programs talk to each other and it worked. Another test with .Net <–> Delphi was succesful too; promising.

Now, but not after I’ve tested some basic needs very thoroughly, I can start dissecting my monolithic system into smaller pieces and gradually create an even more stable, flexible and adaptive Domotica system. One of the upcoming projects will be integration of a Smart Meter for gas and electricity; I think I’ll do that one the 0MQ way – exciting!