Sharing metadata between 2 one-legged systems

The time has come to implement some drivers that need more information than just some low-level information like IP address/port number of the hardware they have to connect to.

For instance, I need an XMLRPC server on the Raspberry Pi, cause XMLRPC  is the method that’s being used by 2 internal User Interfaces (my Philips Pronto TSU9600 and a VB.Net application on a Asus EEE TOP). Those 2 UI’s use XMLRPC to send commands (that eventually end up at the hardware driver) like switching lights, controlling roller shutters, setting temperatures on the radiator valves etc. I don’t want to change that yet, especially on the Pronto, cause it’s a battery powered device where it’s not that smart to keep a network connection open all the time. So I’ll stick with XMLRPC on the Pronto, until I have a better alternative.

The way in which the Pronto sends it commands is like this (a Prontoscript that’s being executed when someone touches the ‘Media’ icon on the Pronto touchscreen):

// UPC box, set channel 1 (to make sure power state is on)
// followed by off // (toggle key!)
XMLRPC_SetDevice('upc','1');
scheduleAfter(5000, function() {
  XMLRPC_SetDevice('upc','off');
});
//Dune hd max power on
XMLRPC_SetDevice('hdmax','on');
// onkyo power on
XMLRPC_SetDevice('onkyo','on');
System.delay(2000);
// onkyo, select HDMI input
XMLRPC_SetDevice('onkyo.input','dvd');
// set default volume level
XMLRPC_SetDevice('onkyo.volume','36');
//TV livingroom on
XMLRPC_SetDevice('LE40B651','on');

The Pronto doesn’t ‘talk’ directly to hardware drivers, but to a device object, using XMLRPC as the transport mechanism – this UI only communicates what has to be done – it doesn’t care about the how..

And that’s good, because sometimes multiple drivers are involved in controlling a piece of hardware. For instance, my media player has a built-in http interface to which I can send commands like play, pause,  stop and so on. But there’s one thing I can not do: turn the device on via this http interface, because it the http interface doesn’t exist while the player is off.. so when the media player is off, I have to send the Infrared code for ‘on’ to the media player with the IRTrans driver. So more than 1 driver can be needed to control a device, even though this is rare.

This kind of knowledge/logic doesn’t belong in a UI, hidden somewhere in a small script, running on the Pronto; the fact that I need to control my media player this way is something that’s hardware specific – it belongs to the devices’ metadata so to speak.

So when the XMLRPC server receives a command like ‘hdmax, on‘, it needs to know more about the hdmax device (i.e. the media player) than just that it exists – it needs to be able to route the command to the device object and the device object will know what to do to have a command actually being executed. In this case: if the command is ‘on‘, forward it to the IRTrans driver, otherwise forward it to the hdmax http driver. That’s roughly how my system works internally. With the XMLRPC server embedded in my Delphi system this wasn’t a problem, cause all the device objects lived in the same memory space as the XMLRPC server – a simple internal message system was all that was needed to get information from A to B.

But with the way my Raspberry Pi based system works, things are more separated. I have an XMLRPC server running as a single process, an IRTrans driver as another single process and the hdmax driver is a separate process as well. And sometimes those processes need information about the devices –  the metadata. And not all those processes need the same metadata…

During the limited time I’ve had during the last few weeks, I began with retrieving the metadata from my MS SQL Server in a way so that it could be used for various purposes. For example, the XMLRPC server has to know some things about devices, their device values and so on. Another process (for example for storing historic data) will need information about what device values to store, the storage interval (number of minutes/hours, minimal, maximum), how long the historic data should be kept in the database, etcetera.

Retrieving information from a MS SQL Server with Node.js was already covered (tedious), so now I needed a way to transform ‘flat’ rows and columns from a SQL database to something that could be handled better in Node.js – I got the idea of creating an Array of device metadata objects, searchable on multiple properties and the ability to define a customized subset of the required metadata per process (for those that need it, cause only a few will) – all nicely tucked away in a module.

First I created an SQL query that would retrieve as much metadata as possible in a single query:

select devices.*, values.*, floors.*, rooms.* from devices
left join values on values.deviceid = devices.deviceid
left join rooms on rooms.roomid = devices.roomid
left join floors on floors.floorid = rooms.floorid
order by address, id

This query results in 1668 rows of 68 columns, containing all the information (metadata) of 285 devices. Most of the time this is way too much, so I added a ‘filter’ so that I could reduce the amount of information, like this:

var coldefs = [];
coldefs[‘DEVICEID’] = {‘name’:’devid’};
coldefs[‘ADDRESS’] = {‘name’:’address’};
coldefs[‘DRIVER’] = {‘name’:’driver’};
coldefs[‘VALUEID’] = {‘name’:’devicevalue’};

Now, only the 4 query result columns specified on the left hand side will be used, the rest of the other 64 columns will be discarded. The remaining ‘metadata result set’ is then transformed into an array of 285 device metadata objects, of which one looks like this:

 { devid: 'dressoir',
   address: 'b02',
   driver: 'plcbus',
   devicevalues: [
    { devicevalue: 'dressoir.signal' },
    { devicevalue: 'dressoir.noise' },
    { devicevalue: 'dressoir.on' }
   ]}

Cause that’s all the XMLRPC driver needs to know – the name (id) used throughout the system, its physical address (in this case a PLCBUS address, but it could just as well be an IP address, MAC address, Visonic sensor address, whatever), the driver used for controlling this device and information about all the values this device has (signal strenth, noise level, on).

With this array of device metadata I can do lots of handy stuff, like:

// this will display "b02":
console.log(metadata.deviceByDeviceValue('dressoir.on').address);

// this will display "plcbus":
console.log(metadata.deviceByDeviceId('dressoir').driver);

// this will display "dressoir":
console.log(metadata.deviceByAddress('b02').devid);

Much better than a Delphi TStringList! 🙂

Although this may look like some next trivial addition to my Node.JS based Home Automation system, it certainly is not. Retrieving information from a MS SQL Server is covered, so I don’t have to worry about ‘databases’ in general anymore for some time, until I’m ready for migrating to (probably even) another type of database. This also means I can continue walking down the migration path as I’ve been doing during the last 2-3 months: gradually rebuilding (and sometimes rethinking) my system.

And now, it’s back to the staircase again!

Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *