Opentherm Monitor finished

This post will be the last one about the Opentherm Monitor. OTOH, when is something really completely finished…

Arduino Serial Monitor

I could spend some more hours on the Opentherm (OT) Monitor and in particular the sketch, but for now it’s good enough. I should add some extra code to validate the OT frame but that would also mean I won’t be able to analyze ‘strange’ frames with unknown Data ID etcetera on my PC. So I’ll leave it as-is for  now. The Opentherm Gateway is waiting ;-)

From what I’ve seen during the last 24 hours, the ‘quality’ of the frames I receive is quite good; somehow there seems to be an invalid frame on the wires every minute or so, and I can’t find out what it is. This same thing happens with the Opentherm Gateway Monitor, so I think both are having the same problem. The Data ID tells me it’s probably an OEM frame…?

OpenTherm Decoder

The Opentherm Decoder running on my PC receives the 4 OT bytes from a serial port and decodes those bytes to something human readable: whether the frame came from the Thermostat or the Boiler, Message type and the meaning of the Data ID. The 16-bit data value (there where you can find the temperatures, pressure and status bits) is not decoded yet; well, it’s all in the Opentherm Protocol documentation, so that should be no problem.

Now I can use this Opentherm Monitor as an additional display near the boiler! The Remeha Calenta already has a rather large display showing stuff like status, water pressure, whether the pump is running, but it doesn’t display flow- and return temperature, control setpoint and I’m sure I can think of some more interesting stuff I wanna see – that’s what the Opentherm Monitor is going to do for me. I already have a 16×4 LCD, so all I have left to do is finding a suitable enclosure, build everything in there and I’m done!

I really liked getting this Opentherm Monitor to work without errors; in fact, getting it to work was more exciting than building it. Learning on the job about ATMega timers, Manchester decoding and programming the whole thing in C from scratch was one big adventure.

The most important references I used were:

And here‘s the sketch- no additional libraries needed, free to use and no guarantees that it will work for you just as well as it does for me. Have fun!

 

Got it!

Yes, yes, YES! I love it :-)

The rethinking of the OpenTherm bit capturing strategy I did a few days ago really did improve things quite a lot, as can be seen on a screendump of my OT Decoder:

"My" OpenTherm decoder

Below is what the Opentherm Monitor, a tool that belongs to the OpenTherm Gateway (yep, I’ve got one of those too, since a week or so ;-) ) is showing in the log:

Gateway Opentherm Monitor

This is great, wonderful result! And the time it took to come this far was well spent, cause I’ve learned a lot in the past days. I’m starting to know the Opentherm Data ID’s by heart, I know some more about the ATMega timers and Manchester decoding has no secrets for me anymore.

You may notice that in my case only the Master (lines with a ‘T’ in them, in both screendumps) frames are decoded and not the Slave response frames (Slave=Boiler, starting with a ‘B’ in the lower screendump), but that’s because I added a delay that causes the sketch to skip the Slave responses.

So the sketch is going to be the next thing I’m going to work on in the next couple of days. But for today, I’m going to relax and and stop thinking about microseconds, timers, prescalers and ISRs;  it’s time for something completely different: a Somfy RS-485 RTS Transmitter for our rolling shutters which will (hopefully) arrive in  a few weeks! That’s why I like Domotica so much – the diversity of things to do and learn.

 

Oops and yeah!

I don’t know what happened, but yesterday’s post is completely wrong. Maybe it was too late in the evening, maybe because I was trying to do 2 things simultaneously,  I don’t know – but the fact is, that yesterday’s post is totally messed up – it doesn’t reflect what I had in my mind at all… so that post was a major f***-up, if I may say so. Well, I’ll fix that some day. Soon, I hope. Maybe I can explain things better, once my thoughts are documented in code ;-)

Fortunately, I didn’t use my own post for what I did today. It’s all in my head, right?  The status so far is that the JeeNode is correctly detecting the short and long periods and it’s reporting those on the Serial port (during the time where there’s no OT communication going on), in a format like this:

FT 1
0S
1S
0S
1L
0S
1S
0L
1L

The FT stands for ‘First Transition’, which tells me that a new transition from 0 to 1 has been spotted.  After that (line 2), the JeeNode input went LOW (0) for a short period (S), HIGH (1) for a short period, LOW for a short period and HIGH for a Long (L) period, …

With this information  I can ‘rebuild’ the signal and the bit stream. For now, I’ve  done this in my favorite programming language Delphi, just to speed things up a bit. The sketch has been running for just half an hour or so and the fact that the reconstructed bit streams have had a length of 68 bits all the time, is encouraging. Cause 68 bits lead to 34 Manchester-decoded bits, which is exactly the size of an OpenTherm frame of 32 bits +1 start- and stop bit. Yeah!!

When I’m 100% sure of how everything should work, I’ll embed all the bit calculations & manipulations  into the sketch, so that all I’ll only receive the resulting 4 bytes per OpenTherm frame from the JeeNode.

But there’s still a lot of work to do before I get there!

Time based sampling vs signal duration

Not being ‘hindered’ by any knowledge about Manchester decoding and signal processing, I came up with the following idea. Rereading the OpenTherm Protocol 2.2 documentation, the following is said about the bitrate and timing:

Bit rate                           : 1000 bits/sec
Period between mid-bit transitions : 900 .. 1150 μs (nominal 1 ms)

Furthermore, there’s a built-in time margin where a transition can take place; When TØ is the start of a bit period of ≈1000 μs, the transition must take place between TØ+400 μs and TØ+650 μs. If not, the transition wouldn’t be conform OT protocol specs. This in fact means that the time window in which the transition can take place is rather large, namely 250 (100+150) μs. That’s 25% of the total bit period, which sounds like a lot to me, actually.

I want to try to code something that’s just as flexible – maybe even more flexible, considering the fact that it’s not unusual for things to be out of spec. Wouldn’t it be better to search for other things that can be measured just as well but which are less dependent on time? Determining long vs. short periods should be enough, maybe? So lets forget about the timing and have a look at the highs and lows for a change. One thing that could be useful is the fact that the signal must always be stable for a period of at least 250 μs; cause if not, it would be out of OT protocol specs.

You could also say that a short period should be between 400 and 650 μs and a long period between 750 and 1250 μs. That means there’s a gap of 100 μs between the longest ‘short period’ and the shortest ‘long period’, but all still within specs. I should be able to determine whether a signal has been stable during a short period or a long one… right?

Update:

The rest of this post has been deleted, because it was totally rubbish and incorrect – what was I thinking?? Too much beer perhaps… ;-)

This will soon be fixed…

Transition timing

Somehow the OpenTherm Monitor I built  a week ago has some problems to correctly decode the signal to Manchester code and produce the correct 32 bits. And the strange thing is, that my logic analyzer which I attached to the input pin on he JeeNode, doesn’t seem to have these problems (click):

BugLogic

The latest version of the Saleae Logic software has Manchester decoding included and the software can make sense of all this, so… Maybe the analyzer is more tolerant? I don’t know.

Time to figure out how ‘good’ the signal really is, and whether it’s worth the time to try to improve the OT Monitor – I don’t just want to give up!

The first thing I did, was having a look at the distribution of the periods between all the transitions (high <–> low). For that I wrote a sketch that used TIMER2 to check the value of the input pin every 80 µs. The time between each transition was measured and stored into an array during sampling. And after a certain number of transitions, it would write the array to the Serial port.

The Serial Output looked something like this:

2 0
3 1
4 1
5 217
6 2795
7 1246
8 23
9 0
10 0
11 16
12 325
13 281
14 17
15 0
16 1
17 0

Copy & paste to Excel and creating a chart revealed the following:

 

Here you see the distribution of all the measured periods between a transition (for both high to low and vice versa) .The numbers on the X-Axis should be multiplied with 80 µs to get the real duration of all the periods. Well, it’s clear that there’s a peak at ≈500 µs and a second one at ≈1000 µs (1 ms); that’s good and what can be expected for a signal with 1 ms bit rate and where the difference in duration between the short and long periods is a factor of 2. So far so good.

Another test I did was to see how the ‘low’ periods and ‘high’ periods related to each other; did they both perform equally well?

Don’t mind the X-axis numbers, I’m not that good with Excel ;-) – it’s the red (high periods) and blue (low periods) lines that count. Again, the time of the second peak (63)  is twice as larga as the first (31..32). That’s good.

But the chart is not that ‘clean’ anymore… some yet unexplainable things popped up. For example,  the leftmost peak of the high periods (e.g. the short high periods) shows a strange curve to the right. In words, this means that there are quite a lot of short high periods that take longer than you’d want – cause the ideal situation would be a chart with only 2 narrow, high peaks for both the highs and lows, right? Could this mess up things? It could very well be that 1 single transition messes up the timing and hence the decoding of a complete OT frame…

I don’t know if that last thought (1 single period messing up things)  is right  - I have to dig some more, understand the OT Monitor sketch, try to find out where in the frame the timing goes wrong and see if I can find a pattern somewhere. If there’s a pattern, maybe I can create a work-around for it to improve the decoding..

Ow, and why’s there a huge dip in the blue (low) chart?

To be continued…

 

Arduino OpenTherm Monitor Shield

Yesterday Freddy Martens contacted me. His name rang a bell, because I had seen his name before.. yep,  he is/has also been working on the Pale Blue Dot Opentherm Monitor as can be seen in the comments at the bottom of the page. He sent me an email and told me that  he had some Eagle designs for an Arduino Shield for the OpenTherm Monitor. His design is based on the schematic he found on the Pale Blue Dot site which I mentioned before – here and in earlier posts. And he asked me if I could host his Eagle files. Sure!

Arduino OpenTherm Monitor Shield

 

The archive contains both a single-layer and a 2-layer design. He also told me that the designs are not perfect – for example, some of the pads are too small. But this could be a good starting point for others. So if you’re interested, just download the OpenTherm4Arduino archive.

Freddy also provided some images he made of his Arduino OpenTherm Monitor shield, which can be seen here.

Have fun!

Current status on my own OpenTherm Monitor project is that I’m getting lots of decoding errors- but I’m not giving up yet!

 

Getting closer

Yesterday the last parts for the OpenTherm Monitor arrived, so I could finish that part last evening:

- Adding the last part to the PCB (a 4.7 Ω resistor);
- Removing a diode that was there to power the PCB from the RS232 port that I won’t use;
- Replace the 2nd “RS232 power” diode with a wire;
- Cut away the unused headers.

And this is the end result:

OpenTherm Monitor PCB finished So the OT Monitor part is finished now. As you can see the DTR connection is now unused and the DSR connection doesn’t have a diode anymore.

Next step was to have a close look at the additional parts that are needed to connect it all to a JeeNode:

Additional parts

On this part of the schematic (here’s the complete original), 5 new parts are being used:

  • 2 resistors of 1 kΩ;
  • 2 * 2N7000 MOSFET (available here)
7N2000 MOSFET

7N2000 MOSFETThese 2 pictures and the the datasheet saying that

pin 1 = Source
pin 2 = Gate
pin 3 = Drain

is enough to make no mistake with the wiring, even if  you have no clue what the 2N7000 does ;-)

 

  • A SN7432N which is a Quadruple 2 input positive OR Gate and can be purchased here

SN7432N

As can be seen in the schematic, a Single 2 input positive OR Gate would suffice just as well; maybe they exist too, but I didn’t bother to search

 

SN7432N powerBesides the 2 inputs and the output signal, the SN7432N als needs to be powered (pin 7 and 14), so the total number of connected pins results in 2+1+2=5.

 

 

 

With a breadboard, a JeeNode, a 4×20 LCD with a JeeLabs LCD Plug on its back, some wires and the SN7432N I mentioned above, this is the end result for now:

 Almost finished

Yep, I’m waiting for the MOSFETs to arrive. As soon as they arrive and I have some time to finish the breadboard, I’m going to put this OT Monitor between the Honeywell Evotouch Wireless OpenTherm RF module and our Remeha Calenta.

Assuming everything keeps working and that I’m not confronted with errors on the display of the boiler, I can try to have a look at the signal on the data pin – and if that looks OK, the next step will be connecting the data pin to the JeeNode and watch the LCD… exciting!

Preparing for OpenTherm Monitoring

More and more my weblog is becoming an ‘archive’ of what I did and, more important, how I did it. I know that I can find that specific picture or other piece of information I’m looking for on my NAS or a local HDD somewhere, but a search on my own weblog is much quicker, and it works too, more and more. So I decided to write down every step I take with this OpenTherm project on my blog and in more detail than I’ve done before – I think I’ll thank myself for that later ;-)

Before heating up the soldering iron, I spent some time reading the OT Protocol Specification v2-2. I won’t go into all the details, but here are some important things to know.

OpenTherm is a point-to-point communication system between boilers and room units (the thermostat). The room unit is supposed to calculate a heat demand which is sent to the boiler; the boiler should act on the information provided (more or less heating) and the boiler can also send back various types of information like diagnostics and other useful stuff (to display on the thermostat LCD, for example).

The physical way in which the boiler and thermostat exchange information over the 2 wires is that the boiler transmits a signal by changing the current (high=17 .. 23 mA, low=5 .. 9mA), while a thermostat does this by changing the voltage (high= 15 .. 18V, low = 7V).  The boiler is also supposed to provide the room unit with power, so no batteries should be needed.

The encoding method that’s being used is called Manchester, where a transition from low to high represents a ’0′ bit value and a transition from high to low is a ’1′; more about Manchester code here. OpenTherm uses a bitrate of 1000 bits/second.

An OpenTherm frame consists of 32 bits e.g. 4 bytes and those 32 bits are transmitted with a leading start- and a trailing stop-bit. These 32 bits contain information like parity, message type (read, write, read-acknowledge, invalid data), data identifier and 1 or 2 (used) data bytes. In a OpenTherm conversation it’s always the thermostat (room unit) that acts as master; hence, the boiler is the slave. Communication is therefore always initiated by the thermostat, not by the boiler and the latter is supposed to reply within 800 ms. Furthermore, the protocol documentation states that a master must communicate at least every second. (question for myself: how does this relate to a wireless connection between thermostat and boiler?)

That’s enough info for now; lets have a look at the first thing I want to do: monitoring OpenTherm traffic with the Elektor OpenTherm Monitor. Lets have a look at the schematic of the Elektor Opentherm Monitor (OT Monitor) and what has to be done to connect this OT Monitor to an Arduino (or a JeeNode, in my case) – cause that’s the goal of this first step.

 

(the image can be a bit blurry due to the resizing; clicking it will reveal a better image)

The right side is where the builder of the OT Monitor is supposed to hook up his/her RS232 port. But that’s not going to happen – at least not if I can manage to modify the Elektor OT Monitor to this schematic, which I found on a site I mentioned before: http://www.palebluedot.nl/jml/projects/arduino/24-openthermmon.html

I can clearly see the modifications on the right side of the schematic (duh), but to make sure I don’t forget or oversee any of the modifications, I opened the original Elektor schematic in MSPaint and started modifying the original to get to the end result that I need. All modifications were done in red, so that it would become much clearer to me what had to be changed on the original PCB and what additional components had to be added (on a separate piece of perfboard):

 

It might look ugly or silly (or both), but it works for me and that’s all that counts, right? It reduces the labyrinth..

So, 1 diode needs to be replaced by a wire and the other connection with a diode will (must) not be used anymore – that takes care of the PCB-side of what has to be done. That should not be too hard.

Low Temperatures & OpenTherm

I don’t know why, but the cold weather of the last few weeks makes the word “OpenTherm” go through my mind all the time… temperatures are in control of what keeps me busy, how about that ;-)

While the Nemef Radaris Plugin is being tested, I have some spare time to try and make that first syllable in OpenTherm a bit more true. At least for me, cause I don’t know that much about the OpenTherm protocol yet – time to do something about it!

First I spent quite a lot of time searching for sources regarding OpenTherm and what I could do to make OpenTherm work for me. Sure, I’m building the OpenTherm Gateway that is described in detail here, but that’s not enough; I want to know more – everything there is to know! The biggest problem I’m facing is my lack of knowledge of electronics. So when it comes down to building stuff to interface with OpenTherm hardware, I have to fall back to schematics made by others. Sad but true.

So I started off with a document I already know about for a long long time (and that anyone who has ever searched the net for information about OpenTherm must have seen before): the Elektuur/Elektor article about the OpenTherm Monitor that was published in 2001. I’ve always wanted to build this Opentherm Monitor, but there’s a tiny problem with this OT Monitor: it needs to be connected to a serial port; not my favorite way of doing this. However, searching some more resulted in this site. Here you can find detailed information on how to connect a slightly modified OT Monitor and some extra components directly to an Arduino; that’s much better, for me.

Another good source of information is Alex Vieira’s blog, which is also mentioned on the JeeLabs Café. Alex’s approach is (as far as I can remember) different from others, cause Alex is using a JeeNode to act as a Thermostat instead of the ‘man in the middle’ approach that is being used by the Opentherm Gateway (injecting/modifying OT frames  between boiler and thermostat) that I’m building.

This made me wonder what would be the best thing to do; yes, I do want to control my central heating and yes, I do want to be able to change the temperature setpoint. But do I want to take over the role of the thermostat all the way and in fact make the current thermostat obsolete? No, cause in my opinion, our central heating system as a ‘basic facility’ that should always stay operational, no matter what. Computers can break down, the additional (DIY) hardware can suddenly stop working by some unforseen reason, or Murpy’s Law kicks in somewhere unexpected – all these things should never affect the service level of the central heating! OK, comfort level will decrease (cause if not, I’m doing something terribly wrong ;-) ), but we won’t have to be afraid of frozen water pipes, cause the ‘real’ thermostat will still keep the house warm enough. And just as important, problems can be fixed very easily by removing the additional hardware and things are back to how it was – as in a ‘normal house’.

You might think I don’t trust a DIY thermostat enough to replace the real one? No, that’s not the issue here – it’s more about creating something that can keep on working forever. Without me, my domotica system or any other piece of hardware I installed. Nothing I do should ever create an unusable house; neither should I be needed to fix things – that says it all.

Back to the original plan again, which is getting to know OpenTherm better.

The first step I made was ordering the OT Monitor @ Eurocircuits. Cost: 24 Euro for the PCB and with some components, the total cost is around 30 Euro.

 

OpenTherm Monitor Some components are still missing, but that won’t take long anymore. Next step will be to add the ‘extras’ I mentioned before so I can connect the OT Monitor to a JeeNode. These ‘extras’ have been ordered a few days ago and a JeeNode is already waiting to display some results on the 16×2 display. Total cost: not much.

Another great thing is that I managed to obtain a PCB out of a very small series that was produced for the OpenTherm Gateway I mentioned above; although I already started building this Gateway on perfboard, I’m happy to start all over again now I have this PCB ;-)

OpenTherm Gateway PCB


To be continued!

Controlling the radiators is not enough

This is the next episode in the never ending story to get a good central heating; not just in the living-room, but in all the rooms; and totally integrated in my Domotica system. Today I received a box full of parts; diodes, resistors, capacitors, a PIC – 33 different parts in total.

A pile of parts

These parts will be used to build this Opentherm Gateway. I think most people who have ever searched for a way to control their Opentherm controlled central heating system will already know the site I referred to, cause it pops up really quick in the search results of any search engine – it’s one of the scarce good resources that are available when you want to integrate your central heating system into your Home Automation system.

But why do I want to control my central heating system? Do we need it, since we’ve already got Radiator Thermostats on all the radiators in our house? Yes we need it, and I’ll explain why.

The biggest “problem” we’re facing is that we have  a single thermostat that’s controlling the central heating right now, and this thermostat is in the living-room. The influence of the sun on the temperature in the livingroom is huge. Today, with a reasonable amount of sunlight, the 6 m2 of glass on the south side of the livingroom result in producing enough energy to warm up the living-room to a temperature above the thermostat setback. That’s good, very good, and we want to keep it that way of course! But the result of this free energy is that our central heating system stops burning around noon, as you can see below (showing the in- and outgoing water temperature of our boiler):

Central heating behavior

The central heating hasn’t been burning from approx. 13:00, cause there was no need for it anymore. There’s no need to explain what happens to the rooms on the 2nd floor; they all cool down, and there’s nothing I can do about it.

Another problem is that the temperature of the water which flows through the radiators, isn’t high enough for the radiators to radiate enough to heat up the rooms upstairs during the time that the boiler does burn. The result of all this: even if we’d want to, the temperatures upstairs will never reach more than 19° Celsius. We don’t need those higher temperatures, but that can change – and it will.

So how can I change when and (more important) where the central heating pumps energy into our house? I decided to give my Domotica system a role in this. The Opentherm Gateway is part of the solution. More about the ‘how‘ later….