If you remember my first post about home automation I mentioned a desire to use some sort of presence detection as part of deciding when to turn the heat on. Home Assistant has a wide selection of presence detection modules available, but the easy ones didn’t seem like the right solutions. I don’t want something that has to run on my phone to say where I am, but using the phone as the proxy for presence seemed reasonable. It connects to the wifi when at home, so watching for that involves no overhead on the phone and should be reliable (as long as I haven’t let my phone run down). I run OpenWRT on my main house router and there are a number of solutions which work by scraping the web interface. openwrt_hass_devicetracker is a bit better but it watches the hostapd logs and my wifi is actually handled by some UniFis.

So how to do it more efficiently? Learn how to watch for ARP requests via Netlink! That way I could have something sitting idle and only doing any work when it sees a new event, that could be small enough to run directly on the router. I could then tie it together with the Mosquitto client libraries and announce presence via MQTT, tying it into Home Assistant with the MQTT Device Tracker.

I’m going to go into a bit more detail about the Netlink side of things, because I found it hard to find simple documentation and ended up reading kernel source code to figure out what I wanted. If you’re not interested in that you can find my mqtt-arp (I suck at naming simple things) tool locally or on GitHub. It ends up as an 8k binary for my MIPS based OpenWRT box and just needs fed a list of MAC addresses to watch for and details of the MQTT server. When it sees a device it cares about make an ARP request it reports the presence for that device as “home” (configurable), rate limiting it to at most once every 2 minutes. Once it hasn’t seen anything from the device for 10 minutes it declares the location to be unknown. I have found Samsung phones are a little prone to disconnecting from the wifi when not in use so you might need to lengthen the timeout if all you have are Samsung devices.

Home Assistant configuration is easy:

device_tracker:
  - platform: mqtt
    devices:
      noodles: 'location/by-mac/0C:11:22:33:44:55'
      helen: 'location/by-mac/4C:11:22:33:44:55'

On to the Netlink stuff…

Firstly, you can watch the netlink messages we’re interested in using iproute2 - just run ip monitor. Works as an unpriviledged user which is nice. This happens via an AF_NETLINK routing socket (rtnetlink(7)):

int sock;
sock = socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE);

We then want to indicate we’re listening for neighbour events:

struct sockaddr_nl group_addr;
bzero(&group_addr, sizeof(group_addr));
group_addr.nl_family = AF_NETLINK;
group_addr.nl_pid = getpid();
group_addr.nl_groups = RTMGRP_NEIGH;
bind(sock, (struct sockaddr *) &group_addr, sizeof(group_addr));

At this point we’re good to go and can wait for an event message:

received = recv(sock, buf, sizeof(buf), 0);

This will be a struct nlmsghdr message and the nlmsg_type field will provide details of what type. In particular I look for RTM_NEWNEIGH, indicating a new neighbour has been seen. This is of type struct ndmsg and immediately follows the struct nlmsghdr in the received message. That has details of the address family type (IPv6 vs IPv4), the state and various flags (such as whether it’s NUD_REACHABLE indicating presence). The only slightly tricky bit comes in working out the MAC address, which is one of potentially several struct nlattr attributes which follow the struct ndmsg. In particular I’m interested in an nla_type of NDA_LLADDR, in which case the attribute data is the MAC address. The main_loop function in mqtt-arp.c shows this - it’s fairly simple stuff, and works nicely. It was just figuring out the relationship between it all and the exact messages I cared about that took me a little time to track down.

This is a public service announcement that the.earth.li (the machine that hosts this blog) will cease service as a Debian mirror on 1st February 2019 at the latest.

It has already been removed from the official list of Debian mirrors. Please update your sources.list to point to an alternative sooner rather than later.

The removal has been driven by a number of factors:

• This mirror was originally setup when I was running Black Cat Networks, and a local mirror was generally useful to us. It’s 11+ years since Black Cat was sold, and 7+ since it moved away from that network.
• the.earth.li currently lives with Bytemark, who already have an official secondary mirror. It does not add any useful resilience to the mirror network.
• For a long time I’ve been unable to mirror all release architectures due to disk space limitations; I think such mirrors are of limited usefulness unless located in locations with dubious connectivity to alternative full mirrors.
• Bytemark have been acquired by IOMart and I’m uncertain as to whether my machine will remain there long term - the acquisition announcement focuses on their cloud service rather than mentioning physical server provision. Disk space requirements are one of my major costs and the Debian mirror makes up ⅔ of my current disk usage. Dropping it will make moving host easier for me, should it prove necessary.

I can’t find an exact record of when I started running a mirror, but it was certainly before April 2005. 13 years doesn’t seem like a bad length of time to have been providing the service. Personally I’ve moved to deb.debian.org but if the network location of the is the reason you chose it then mirror.bytemark.co.uk should be a good option.

My first attempt at home automation involved a dark internal staircase in my flat, a set of white LED fairy lights and a plan to make them switch on when I was at home and the sun had set. I purchased a set of Energenie sockets and the associated Pi-Mote to control the lights and looked at what my control options were. Nothing really stood out so I started writing some Python that would watch for my phone being connected to the wifi, look at whether the sun had set and send the appropriate commands. Unfortunately the range on the Energenie sockets and/or the Pi-mote ended up too poor to work from where the Pi was located to the socket out in the hallway. I tried soldered an antenna onto the Pi-mote, but it still ended up too unreliable. Subsequently something happened to the Pi-mote and it stopped working entirely. The sockets were still working ok, and for a while I made some use of them with the provided remote control, but until recently they had been sitting in a box for a couple of years.

With my more recent, more successful foray into automation I decided to try and revive the Energenie sockets as part of my setup. Poking the Pi-mote confirmed it was dead - trying to drive it manually rather than with the Pi resulted in no output signal. However I’d purchased a cheap 433MHz receiver/transmitter pair (MX-RM-5V + FS1000A) - very cheap and nasty, but easy to poke. I found a good writeup of the Energenie protocol and how to drive it via FT232R bitbanging and Glen Pitt-Pladdy also had some details. Using PulseView and my cheap Cypress FX2 analyser hooked up to the MX-RM-5V I was able to capture the 20 digit code from my remote control.

All that remained was to try and transmit the code + appropriate power on/off commands for each of my 4 sockets. I still have spare Digispark clones, so that seemed like a good starting point. My initial first steps with ATTinys involved a relay board which was available with up to 8 ports. It made sense to try and emulate a 4 port relay board on a device that would actually send out the appropriate 433MHz signals. In addition the relay boards have 5 digits of configurable serial number - perfect for a 20 bit ID code! I took my existing code and fixed it up so that instead of setting/clearing the appropriate output pin bit it wiggled a single pin, connected to the FS1000A, in the appropriate manner for the Energenie. It took a little bit of twiddling, and I had to solder an antenna onto the FS1000A to get better range, but I’ve ended up with 2 sockets in the same room as the Digispark/transmitter dongle, and another in the next room on just the other side of the wall.

For initial testing I used Darryl Bond’s usbrelay, which is handy because it also supports setting the serial number, so I didn’t have to hard code anything into my image. Ultimately I wrote my own code to control the device in Python, and of course hooked it into my MQTT setup. This tied it into Home Assistant like any other set of MQTT lights, and ultimately into Alexa.

Ultimately the SonOff is better technically - the use of an ESP8266 directly on the device means you get a direct secure MQTT/TLS connection rather than an easily sniffable/clonable 433MHz signal, plus it’s bidirectional so you can be sure the device is in the state you think it is. However I had these switches lying around and a spare Digispark so the only expenditure was a couple of quid for the transmitter/receiver pair. Plus it was fun to figure it out and end up with a useful result, and some of the information learned will be useful for controlling my heating (which is on a 433MHz thermostat).

The code is locally and on GitHub in case it’s of use/interest to anyone else.

I’m just back from DebConf18, which was held in Hsinchu, Taiwan. I went without any real concrete plans about what I wanted to work on - I had some options if I found myself at a loose end, but no preconceptions about what would pan out. In the end I felt I had a very productive conference and I did bits on all of the following:

• Worked on trying to fix my corrupted Lenovo Ideacentre Stick 300 BIOS (testing of current attempt has been waiting until I’m back home and have access to the device again, so hopefully within the next few days)
• NMUed sdcc to fix FTBFS with GCC 8
• Prepared Pulseview upload to fix FTBFS with GCC 8, upload stalled on libsigc++2.0 (Bug#897895)
• Caught up with Gunnar re keyring stuff
• Convinced John Sullivan to come and help out keyring-maint
• New Member / Front Desk conversations
• Worked on gcc toolchain packages for ESP8266 (xtensa-lx106) (Bug#868895). Not sure if these are useful enough to others to upload or not, but so far I’ve moved from 4.8.5 to 7.3 and things seem happy.
• Worked on porting latest newlib to xtensa with help from Keith Packard (in particular his nano variant with much smaller stdio code)
• Helped present the state of Debian + the GDPR
• Sat on the New Members BoF panel
• Went to a whole bunch of interesting talks + BoFs.
• Put faces to a number of names, as well as doing the usual catchup with the familiar faces.

I managed to catch the DebConf bug towards the end of the conference, which was unfortunate - I had been eating the venue food at the start of the week and it would have been nice to explore the options in Hsinchu itself for dinner, but a dodgy tummy makes that an unwise idea. Thanks to Stuart Prescott I squeezed in a short daytrip to Taipei yesterday as my flight was in the evening and I was going to have to miss all the closing sessions anyway. So at least I didn’t completely avoid seeing some of Taiwan when I was there.

As usual thanks to all the organisers for their hard work, and looking forward to DebConf19 in Curitiba, Brazil!

Having setup a central MQTT broker I’ve wanted to feed it extra data. The study temperature was a start, but not the most useful piece of data when working towards controlling the central heating. As it happens I have a machine in the living room hooked up to the TV, so I thought about buying something like a TEMPer USB so I could sample the room temperature and add it as a data source. And then I realised that I still had a bunch of Digispark clones and some Maxim DS18B20 1-Wire temperature sensors and I should build something instead.

I decided to try and emulate the TEMPer device rather than doing something unique. V-USB was pressed into service and some furious Googling took place to try and find out the details of how the TEMPer appears to the host in order to craft the appropriate USB/HID descriptors to present - actually finding some lsusb output was the hardest part. Looking at the code of various tools designed to talk to the device provided details of the different init commands that needed to be recognised and a basic skeleton framework (reporting a constant 15°C temperature) was crafted. Once that was working with the existing client code knocking up some 1-Wire code to query the DS18B20 wasn’t too much effort (I seem to keep implementing this code on various devices).

At this point things became less reliable. The V-USB code is an evil (and very clever) set of interrupt driven GPIO bit banging routines, working around the fact that the ATTiny doesn’t have a USB port. 1-Wire is a timed protocol, so the simple implementation involves a bunch of delays. To add to this the temper-python library decides to do a USB device reset if it sees a timeout. And does a double read to work around some behaviour of the real hardware. Doing a 1-Wire transaction directly in response to these requests causes lots of problems, so I implemented a timer to do a 1-Wire temperature check once every 10 seconds, and then the request from the host just returns the last value read. This is a lot more reliable, but still sees a few resets a day. It would be nice to fix this, but for the moment it’s good enough for my needs - I’m reading temperature once a minute to report back to the MQTT server, but it offends me to see the USB resets in the kernel log.

Additionally I had some problems with accuracy. Firstly it seems the batch of DS18B20s I have can vary by 1-2°C, so I ended up adjusting for this in the code that runs on the host. Secondly I mounted the DS18B20 on the Digispark board, as in the picture. The USB cable ensures it’s far enough away from the host (rather than sitting plugged directly into the back of the machine and measuring the PSU fan output temperature), but the LED on the board turned out to be close enough that it affected the reading. I have no need for it so I just ended up removing it.

The code is locally and on GitHub in case it’s of use/interest to anyone else.

(I’m currently at DebConf18 but I’ll wait until it’s over before I write it up, and I’ve been meaning to blog about this for a while anyway.)