If you know me, you know that I must amass data. To that end, I knew my water meter broadcast my water usage, and I had experimented with receiving that data in the past. I also recently discovered this cool robot you can attach to any quarter turn water valve and remotely shut off your water. I figured I could start reading and storing my water meter data, and use that to track usage, and also to attempt to detect leaks by looking for water usage while we were away or asleep.
As is tradition, we have a long journey to get there.
Smart Meters
A smart meter is a meter in which the utility can read it without sending a person to walk up to it and read it with their eyes, or at least that's the definition I'm going to use here. The newest smart meters are connected to the internet or form their own mesh network and the utility can get the readings on demand, whenever they want (called Advanced Metering Interface, or AMI). There is also an older type of smart meter which simply broadcasts their readings every few minutes (called Automatic Meter Reading, or AMR). The utility can then send someone around in a car to read all the meters while driving by. They still need to send people out to all the neighborhoods, but they can cover way more ground than when they had to walk up to each meter to read it.
While the broadcast type of smart meter sounds worse, it's better for those of us who wish to hack into the signal and automate collecting data from their meters. AMI meters are likely to be encrypted and the only way you can really access the data is if the utility provides a cloud based API to get it. Many do, but I always prefer local solutions over cloud solutions. AMR meters are (generally) unencrypted, and all we need to read them is something that will receive and understand the radio signals.
Software Defined Radio (SDR)
Software Defined Radio (SDR) is just what it sounds like. It's a radio in which you can write software to determine how it functions and reads radio signals. You get a hardware dongle and antenna and then can use different software to receive basically any radio signal. The RTL-SDR is a very common open source one. Everything you need is included in a kit for about $40.
Software
There is a ton of already written software out there for just about any purpose you could want to use a SDR for. For reading smart meters the most common one is rtlamr. The readme on that page is pretty good, but you run a separate utility called rtl_tcp in one console, then you run rtlamr in another console. Then it'll begin dumping every smart meter packet it receives in your area.
Finding the right meter
If you can see your neighbors' houses then you'll be able to also read their meters when you fire up rtlamr. Figuring out which meter was mine was my first stumbling block. The ID of your meter should be printed either on the meter itself, or on a box that is wired to the meter. I have three devices with serial numbers on them that are all connected together. I copied down any serial numbers written on them and the current reading of my meter, and began looking through the data for a meter that matched either the serial numbers or reading. There were at least a dozen meters I was reliably picking up, with a dozen more that I'd get occasional signals from. I figured mine would be one of the strongest signals, and therefore one of the most frequent in the list. The meters all seemed to broadcast every minute or so.
Still, none of the meters seemed to match mine. This is what I was seeing (slightly anonymized):
You can see it lists Type 5 and 12. This file says that type 5 is electrical, and type 12 is either gas or electrical. That was odd, because I didn't think I had a smart electric meter, and thought I probably had a smart gas meter, but was just about certain I had a smart water meter.
This is one of the boxes that was attached to my water meter. Googling it showed that a Neptune R900 was a very common water meter broadcaster. I wondered if the ID it was broadcasting simply wasn't on the box, and if the reading it was broadcasting had some offset vs the one on my physical meter. I had the candidate meters limited to a handful which I seemed to always receive the signals of, but none matched up.
Gameshark
Allow me to take you on a tangent. This process reminded me a lot of my N64 Gameshark, which I often credit with getting me really interested in computers. A Gameshark was a device a video game cartridge plugged into and then it, in turn, plugged into the console. With the Gameshark between the game and the console it could access all the game's memory, reading and changing any values you wanted. This let you cheat in games, but first you had to know what memory addresses to change, and how. The memory addresses were always the same for a given game, but would be different for different games (or versions of the same game). So if you wanted infinite lives in Mario 64, that was stored in some memory address, and once you knew which one it was, you (or anyone else who knew that address) could set that lives counter to whatever you want. You could, of course, look these codes up online, but this was in an era of the internet when not literally every known piece of information was available within seconds. It was often hard to find the codes you wanted, particularly for newer, or more obscure games, or if you wanted to do something else besides the common things like "infinite lives".
Earlier devices, like Game Genie, had a book with codes in it, but for new games you were out of luck. The Gameshark provided a DIY way of getting these codes. There was a button on the Gameshark, and when you pressed it, it froze the game and brought up a special menu. This menu let you start a search for the right memory addresses that contained whatever it was you wanted to modify. You then resumed the game and played for a bit. Then you went back to that menu and told the Gameshark how the value you are searching for had changed. You could say it went up, went down, stayed the same, or give it an exact value, or just that it changed in any way. You repeated this process a few times, and eventually, from the millions of memory addresses only one would have changed in the exact pattern you specified. Then you knew you had the right one, and you could change its value.
This process was super interesting to my 14 year old self. The idea that literally everything about this game was just values stored in memory, and that I could modify any of them and that would modify the game I was playing was magic.
Finding the right meter, part 2
Anyway, with that section covered, I can now use this sentence: Like a real life Gameshark, I spent the night with my laptop next to my water meter and utility sink, using water to chage the water meter reading and waiting to see which broadcast meters matched my changes. However, this only lead to more confusion as no meter seemed to match up with what I was doing. This was complicated by not knowing what kind of lag there was between when water was used and when the new value would be broadcast, but I had pretty much ruled out any of the meters I could read being my water meter.
Upon researching this, I eventually stumbled upon some examples of other people reading R900 water meters with rtlamr. They were all using the r900 type flag when launching rtlamr, rather than the default scm type I had been using. For some reason I had assumed the default type would pick up all meters, and hadn't really paid attention to the other types. Sure enough, changing the command to use the r900 type I got a totally different set of meters, with different types of data. One of those meters had the exact ID printed on the yellow label on my Neptune R900 box, and so I was sure I had found the right meter.
Getting the data into Home Assistant
I knew I wanted to use MQTT as an intermediary to get the data into Home Assistant (HA). When I first started dabbling with DIY sensors for HA I wrote a few custom components which would show up as a sensor directly in HA. This proved to be a lot of work, particularly early on there were a lot of changes to the way these components had to work, and it was clear that making a custom component was intended for people who were planning on releasing their component for others to use. I had no intentions of ever doing that, and having to support random people trying to use whatever hacked together, super specific to me, nonsense I had whipped up.
After a while, I just started writing data directly to the database. Which I still do for a few things. It works fine, although the one catch is I generally only put the data into the history table so it shows up in the history graphs. I don't have the values available as a sensor that I can use to drive automations. This is fine for most my purposes, because tracking the data is what I'm mostly after.
However, all of that is moot now, because I've discovered how easy MQTT is to use for this stuff. MQTT is a protocol for sending messages on your local network, and is frequently used for DIY smart home devices and sensors. You set up a server to run as a "broker" on your network, and then all your devices can publish to or subscribe to "topics". So my DIY temperature sensors will publish to topics like: "home/bedroom/temperature" with a JSON payload there. Home assistant then has a custom sensor you can use to read any topic and turn the values on it into a sensor. If you get into MQTT I recommend MQTT Explorer as a GUI to help watch what data is being published to topics on your local MQTT network.
I almost began writing something to parse the text from this command line utility and send it to MQTT, but figured this would be the type of thing with a lot of glitches, and had the sense to search to see if someone else had already dealt with the headaches and edge cases. Sure enough, I found a Python script called rtlamr2mqtt which did exactly what I wanted. In fact, it even had an option to use the Home Assistant autodiscovery feature where, if you correctly format the MQTT topic, HA will just pick it up automatically.
The project uses Docker, but as I refuse to learn Docker, I just followed the steps in the Dockerfile manually and installed it as a regular ol' python script.
This was pretty much it. There were some minor tweaks to the .yaml file that contained the configuration. One cool thing I did was figure out I could multiple my readings by 10 by adding a 0 to the end of the format string, like this: "#######0". This was nice because I was pretty sure my readings were for 10s of gallons. The reading I was picking up didn't line up at all with what was on the physical meter, but I never really thought they would. All that matters was if they moved by the same amount.
Over time though, I discovered the readings I was getting did not scale with what was on the physical meter at all. They would for a short time, but then jump ahead, seemingly at random. But, that odyssey will be in part 2.
No comments:
Post a Comment