Suspicious discontinuities

https://danluu.com/discontinuities/

This page is a wall of text and some charts, but it shows some really interesting patterns in a wide array of data sets.

How I DIY smart home sensors

Home Assistant

I've had a draft post about Home Assistant for years, which I haven't posted because I feel like my HA install is perpetually under construction.  If you haven't heard of Home Assistant, it's a LAN based, open source, smart home control center.  Typically installed on a Raspberry Pi, it has an absurd number of integrations with third parties.  While the UI of HA can be polished up to be pretty nice, its real value is in automatons that never require opening the UI.  This isn't a HA post so I'll stop talking about it, but I highly recommend playing around with it if you have any interest in smart home, and are willing to tinker a bit in exchange for not having to use cloud based systems.

Sensors

While I have a variety of sensors that serve particular purposes, I'm obsessed with collecting basic data for each area of my house.  Particularly, I like to measure the temperature, humidity, and light level on each level on my house (including attic and basement).  I do nothing with this information, but I look at the graphs literally every day, and they bring me great joy.  If you're confused right now, this is probably a good time for you to cut your loses and stop reading this post.  If you're excited for your own graphs, read on.

Data

While you can buy ready made sensors that measure these things, I've always built my own.  They are much cheaper, and you have way more flexibility if you decide you want to add something later.  Originally I started out with Raspberry Pis, and then Rasperrry Pi Zero Ws.  This works, but if you start wanting to do multiples of these, then the overheads of SD cards and high current power supplies start to add up.

I've always considered using an ESP32 (what an Arduino is based on) instead, but they are just more annoying to work with.  While a Raspberry Pi is a full computer, running Linux, an ESP32 is a microcontroller, which means you write a single program (typically in C), compile it and load it up, and when it powers up it runs that program, and does nothing else.

 

ESPHome

While the hassle of the ESP32 has never been worth the cost savings over the Raspberry Pi for me, this all changed when I discovered a project called ESPHome.  With ESPHome you connect some sensors to an ESP32, create a YAML file with the config, then upload the files to the ESP32, which will then begin sending the data to Home Assistant via an integration.  I couldn't believe how slick the setup was.  You can even update the config file and send the update to the ESP32 via WiFi without having to go to wherever you have it installed and hooking it up to a computer.  With ESPHome, setting up these sensors is easier with ESP32s than my custom Python scripts running on Raspberry Pis.

Hardware

These links are mostly dead, as these listing change often.  But, I'll provide them, and the price I paid, as a reference for you, visitor from the future.

ESP32 ($6.50 each) - There are a few versions of these out there, and I'm not an expert on the differences, but for our purposes I don't think they matter much.  Just note what pin layout you get, and make sure when you buy more they are the same layout.

BME280 ($24 / 3) - Temperature, humidity, and pressure sensors.  I used to use DHT22 sensors, but BME280s are: 1. more accurate, 2. use the I2C protocol, and 3. include pressure.  I can put two DHT22 sensors next to each other and while it's obvious the graphs follow each other, there is clear variation between them.  With two BME280s the graphs are spot on.  They are also much higher resolution, which makes the graph much smoother.  The I2C protocol allows you to share one pin across many sensors.  Each sensor gets an address and the host will cycle through them measuring each one.  This makes it quite easy to add another (I2C based) sensor later on.

BH1750 ($7 / 3) - Light sensor; also uses I2C.

USB Charger - I won't provide a link and cost for these.  The power requirements for ESP32s is much lower than a Raspberry Pi, under 500 mA vs at least 1000 mA to over 2000 mA for newer Raspberry Pis.  You should be able to use any random USB charger you have available.

Prototype Boards ($12 / 40) - Not required.  Used to hold multiple sensors together and wire everything up.

Headers ($14 / 120) - Used with the prototype board.

Jumper / Dupont wires ($7 / 120) - Used to connect sensors.  You can technically get by without these if you use the prototype boards, or you can use just these instead of the boards.

Build

Full disclosure: This isn't going to be the type of guide you can just follow along step by step and end up with something that works.  You'll have to understand what you're actually doing to get this to work.  This isn't intentional on my part, it's just the best I can do explaining this.  If you've soldered a bit before, and are familiar with wiring things up to Pis and Arduino you should be able to get it to work.

My goal is to build a "shield" from a prototype board, which will sit on top of the ESP32 and allow multiple I2C sensors to be plugged into it.  Since I2C generally use the same pin layout, the front of the board will have multiple header sockets aligned vertically, all connected to the same wires on the back of the board.  Those wires will go to 4 pins on the ESP32.  I use a few inches of solid core Ethernet cable to provide the wires.

This is the pinout of the style ESP32 I use.  There are a few common ones, and you have to find the one that matches yours.  The pins we care about are 3V3, GND, I2C SCL, and I2C SDA.  All the sensors I buy have pins in the order VCC, GND, SCL, and SDA, this is common, but something to check when searching.  It's not required to have the pins in this order, but if all your sensors have pins in the same order you can create a bus on the board where all the pins are wired in parallel to the 4 pins of the ESP32.

The way I wire the boards up is going to be hard to explain, but hopefully these pictures help.

Front of the prototype board

Back of the prototype board

Start on the front of the board with two 4 pin sockets, aligned vertically.  On the back of the board, wires are soldered up to the pins of those sockets, and run vertically down the board.  It's not clear in the picture, but there are exactly 8 solder points on the back of the board, for the two 4 pin sockets.  Each of the pins of the two sockets is directly connected to the same pin on the other socket.

At the bottom of the board, the wires run through the board, back to the front.  Looking at the front of the board again, the wires then run all over the board to where the pins are on the ESP32.  If you're using a different ESP32 pinout, the exact layout of the wires on the front will look different.  There are also 8 solder points on the front of the board.  Four of them are the electrical connections to the four pins of the ESP32. and the other four are just physical connections holding the top and bottom of each 20 pin header in place, those are technically optional.

Your ESP32 will then have two male headers soldered to it (possibly by you), which will then plug into the 20 pin female header on the prototype board.  Four pin male headers on the sensors can then be plugged directly into the four pin female headers on the board.  However, I have found using a short 4 wire jumper cable between the BME280 and boards will help keep the heat from the ESP32 from influencing the temperature reading of the BME280.  You could also just subtract an offset in software if you'd rather a neater look.  It's not pictured, but I use masking tape to cover all the exposed wires to prevent shorts and class it up a bit.

Software

The software step is way easier than the hardware.  First, follow the ESPHome getting started guide.  Then, when you have your YAML file, modify the following example to get it working with the BME280 and BH1750:

esphome:
  name: rec_room
  platform: ESP32
  board: esp32doit-devkit-v1

wifi:
  ssid: "MyWifiNetwork"
  password: "Password123"

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Rec Room Fallback Hotspot"
    password: "random_password_abc123"

captive_portal:

# Enable logging
logger:

# Enable Home Assistant API
api:

ota:


i2c:
  sda: 21
  scl: 22
  scan: True

sensor:
  - platform: bh1750
    name: "Rec Room Light"
    address: 0x23
    measurement_time: 254
    update_interval: 60s
    unit_of_measurement: lux

  - platform: bme280
    temperature:
      name: "Rec Room Temperature"
      oversampling: 16x
      filters:
      - lambda: return x * (9.0/5.0) + 32.0;
      unit_of_measurement: "°F"
    pressure:
      name: "Rec Room Pressure"
    humidity:
      name: "Rec Room Humidity"
    address: 0x76
    update_interval: 60s

 

Summary

I'll admit, that when I list this whole process out like this, it's a lot more work than I thought.  It's also not particularly cheap for each sensor package, at just under $20 each.  But it's hard to find sensors that plug in to the wall, instead of using batteries.  That could be a pro or a con to you, but I hate batteries.  These are also really rock solid reliability, and provide super clean data.  Plus, it's fun?

Feynman: FUN TO IMAGINE

This is an old video, but this is a new HD rip from the 16mm film.  I really recommend watching it if you haven't ever.

https://www.youtube.com/watch?v=nYg6jzotiAc

Radio Transmissions From Police Helicopter's Chase Of Bizarre Craft Over Tucson Add To Mystery

https://www.thedrive.com/the-war-zone/42021/radio-transmissions-from-police-helicopters-chase-of-bizarre-craft-over-tucson-add-to-mystery

On February 9, 2021, a U.S. Customs and Border Protection (CBP) helicopter encountered what was described as a “highly modified drone” hovering in controlled airspace above Tucson, Arizona. A Tucson Police Department (TPD) helicopter was called in to aid the CBP aircraft in its pursuit of the small aircraft, but the drone, or whatever it was, was able to outrun both of them as it flew through military airspace, deftly maneuvered around both helicopters with bizarre agility, and ultimately disappeared into cloud cover above the altitude the helicopters could safely fly. A police report previously obtained by The War Zone showed that the TPD crew described the drone as “very sophisticated / specialized” and “able to perform like no other UAS” they had previously encountered. Now we have the actual audio from the CBP helicopter’s interactions with air traffic controllers in Tucson during the incident, as well as audio from an after-action call between the TPD crew and the air traffic control tower.

How many Perseids will I see in 2021?

 https://blogs.nasa.gov/Watch_the_Skies/2021/08/09/how-many-perseids-will-i-see-in-2021/?linkId=127090324

 

Where Are The Robotic Bricklayers?

https://constructionphysics.substack.com/p/where-are-the-robotic-bricklayers

There seems to be a few factors at work. One is the fact that a brick or block isn’t simply set down on a solid surface, but is set on top of a thin layer of mortar, which is a mixture of water, sand, and cementitious material. Mortar has sort of complex physical properties - it’s a non-newtonian fluid, and it’s viscosity increases when it’s moved or shaken. This makes it difficult to apply in a purely mechanical, deterministic way (and also probably makes it difficult for masons to explain what they’re doing - watching them place it you can see lots of complex little motions, and the mortar behaving in sort of strange not-quite-liquid but not-quite-solid ways). And since mortar is a jobsite-mixed material, there will be variation in it’s properties from batch to batch.

Zip - How not to design a file format.

 https://news.ycombinator.com/item?id=27925393

> How do you read a zip file?

> This is undefined by the spec.

> There are 2 obvious ways.

> 1. Scan from the front, when you see an id for a record do the appropriate thing.

> 2. Scan from the back, find the end-of-central-directory-record and then use it to read through the central directory, only looking at things the central directory references.

I was recently bitten by this at work. I got a zip from someone and couldn't find inside the files that were supposed to be there. I asked a colleague, and they sent me a screenshot showing that the files were there, and that they didn't see the set of files that I saw. I listed the content of the zip using the "unzip -l" command. They used the engrampa GUI. At that point I looked at the hexdump of the file. What caught my eye was that I saw the zip magic number near the end of the zip, which was odd. The magic number was also present at the beginning of the file. At this point I suspected that someone used cat(1) to concatenate two zips together. I checked it with dd(1), extracting the sequence of bytes before the second occurrence of the zip magic number and the remainder into two separate files. And sure enough at that point both "unzip -l" and "engrampa" showed the same set of files, and both could show both zips correctly. Turns out engrampa was reading the file forwards, whereas unzip was reading the file backwards. 

‘Scared to Death’ by Arbitration: Companies Drowning in Their Own System

 https://www.nytimes.com/2020/04/06/business/arbitration-overload.html

The hair-dryer incident

 https://slatestarcodex.com/2014/11/21/the-categories-were-made-for-man-not-man-for-the-categories/

The Hair Dryer Incident was probably the biggest dispute I’ve seen in the mental hospital where I work. Most of the time all the psychiatrists get along and have pretty much the same opinion about important things, but people were at each other’s throats about the Hair Dryer Incident.

Basically, this one obsessive compulsive woman would drive to work every morning and worry she had left the hair dryer on and it was going to burn down her house. So she’d drive back home to check that the hair dryer was off, then drive back to work, then worry that maybe she hadn’t really checked well enough, then drive back, and so on ten or twenty times a day.

It’s a pretty typical case of obsessive-compulsive disorder, but it was really interfering with her life. She worked some high-powered job – I think a lawyer – and she was constantly late to everything because of this driving back and forth, to the point where her career was in a downspin and she thought she would have to quit and go on disability. She wasn’t able to go out with friends, she wasn’t even able to go to restaurants because she would keep fretting she left the hair dryer on at home and have to rush back. She’d seen countless psychiatrists, psychologists, and counselors, she’d done all sorts of therapy, she’d taken every medication in the book, and none of them had helped.

So she came to my hospital and was seen by a colleague of mine, who told her “Hey, have you thought about just bringing the hair dryer with you?”

And it worked.

She would be driving to work in the morning, and she’d start worrying she’d left the hair dryer on and it was going to burn down her house, and so she’d look at the seat next to her, and there would be the hair dryer, right there. And she only had the one hair dryer, which was now accounted for. So she would let out a sigh of relief and keep driving to work.

And approximately half the psychiatrists at my hospital thought this was absolutely scandalous, and This Is Not How One Treats Obsessive Compulsive Disorder, and what if it got out to the broader psychiatric community that instead of giving all of these high-tech medications and sophisticated therapies we were just telling people to put their hair dryers on the front seat of their car?

But I think the guy deserved a medal. Here’s someone who was totally untreatable by the normal methods, with a debilitating condition, and a drop-dead simple intervention that nobody else had thought of gave her her life back. If one day I open up my own psychiatric practice, I am half-seriously considering using a picture of a hair dryer as the logo, just to let everyone know where I stand on this issue.

River Runner: drop a raindrop anywhere in the USA, watch where it ends up

 https://river-runner.samlearner.com/