Wednesday, August 14, 2019

Build a computer out of NAND gates in stages.  This is essentially a game version of my post about how computers work.

Sunday, July 7, 2019

Social Science Research Network

 I've been into reading random papers from SSRN lately.  There's some really good stuff on there, like the paper I mentioned in my last post.

Sunday, June 30, 2019

The law of small numbers

I was listening to a podcast when I heard about an interesting probability result in the same vein as the Monty Hall Problem.  The new problem is this: Flip a coin 100 times and record the results.  Now pick random flips in the set and see if the next 3 flips are all heads; if so we call this a streak.  Repeat until you find a streak of 3.  Now what is the probability that the 4th flip is also heads?  Is it 50% like we would expect?  It turns out to be closer to 46%, which is not very far from 50%, but is also a clear trend.

You can download the paper here, and I recommend you read through the introduction, which is pretty easy to follow.  I think does a good job of explaining what is going on.  Since no one will do that, here is a table from the paper which helps give some intuition.

This represents every possible outcome from flipping a coin 3 times and looking for a 'streak' of 1 heads.  There are eight total possible outcomes, all equally likely.   In the first two, the streak of 1 heads never happens, or happens on the last flip where there is no following flip to look at.  Those are thrown away and ignored.  In the other six possible outcomes we do get a streak, at least once, and earlier than the last flip.  The underlined flips represent the possible candidates for the flip that is following a streak.  If we pick the preceding streak, then the underlined flips will be the one we are trying to predict.  In three out of the six outcomes with a streak, the following flip will not be heads.  In two out of the six outcomes the following flip will always be heads.  And in the remaining possible outcome it could be either head or tails with 50/50 probability depending on which streak you pick.

If you list out all the possible outcomes from any combination of streak length and total flips, you can see that some number of the heads flips are 'consumed' by the streaks themselves.  Those flips can never be following a streak, because they are part of the streak needed to define the streak.  On the other hand, the tails have no restrictions, they are all available to occur in the flip immediately following a streak.  There are simply more tails available to go in the candidate position.  The effect gets smaller as you decrease the streak length or increase the total number of flips in a set.

I found this very surprising, so I wanted to test it out.  I wrote a Ruby script to simulate various coin flips and look for streaks of different lengths, and output the results.  I then decided to rewrite it in a compiled language so it would be faster.  I decided to try out Go, as I've never used it before and I was hoping for something with a bit more syntactic sugar than C.

Here are the results of a bunch of combinations of streak lengths and numbers of flips from the Go program:
Looking for a streak of length  1 in    10 total flips. Performed 10000 rounds, and   9973 were successful, found 45.29% continued the streak.
Looking for a streak of length  1 in   100 total flips. Performed 10000 rounds, and  10000 were successful, found 49.43% continued the streak.
Looking for a streak of length  1 in  1000 total flips. Performed 10000 rounds, and  10000 were successful, found 49.91% continued the streak.
Looking for a streak of length  2 in    10 total flips. Performed 10000 rounds, and   8203 were successful, found 38.16% continued the streak.
Looking for a streak of length  2 in   100 total flips. Performed 10000 rounds, and  10000 were successful, found 47.72% continued the streak.
Looking for a streak of length  2 in  1000 total flips. Performed 10000 rounds, and  10000 were successful, found 50.15% continued the streak.
Looking for a streak of length  3 in    10 total flips. Performed 10000 rounds, and   4797 were successful, found 34.88% continued the streak.
Looking for a streak of length  3 in   100 total flips. Performed 10000 rounds, and   9995 were successful, found 45.84% continued the streak.
Looking for a streak of length  3 in  1000 total flips. Performed 10000 rounds, and  10000 were successful, found 49.78% continued the streak.
Looking for a streak of length  4 in    10 total flips. Performed 10000 rounds, and   2152 were successful, found 35.83% continued the streak.
Looking for a streak of length  4 in   100 total flips. Performed 10000 rounds, and   9637 were successful, found 40.61% continued the streak.
Looking for a streak of length  4 in  1000 total flips. Performed 10000 rounds, and  10000 were successful, found 49.21% continued the streak.
Looking for a streak of length  5 in    10 total flips. Performed 10000 rounds, and    985 were successful, found 37.36% continued the streak.
Looking for a streak of length  5 in   100 total flips. Performed 10000 rounds, and   7860 were successful, found 38.66% continued the streak.
Looking for a streak of length  5 in  1000 total flips. Performed 10000 rounds, and  10000 were successful, found 48.91% continued the streak.
Looking for a streak of length  6 in    10 total flips. Performed 10000 rounds, and    388 were successful, found 35.82% continued the streak.
Looking for a streak of length  6 in   100 total flips. Performed 10000 rounds, and   5190 were successful, found 35.24% continued the streak.
Looking for a streak of length  6 in  1000 total flips. Performed 10000 rounds, and   9996 were successful, found 46.68% continued the streak.
Looking for a streak of length  7 in    10 total flips. Performed 10000 rounds, and    140 were successful, found 40.71% continued the streak.
Looking for a streak of length  7 in   100 total flips. Performed 10000 rounds, and   2997 were successful, found 33.83% continued the streak.
Looking for a streak of length  7 in  1000 total flips. Performed 10000 rounds, and   9761 were successful, found 42.40% continued the streak.
Looking for a streak of length  8 in    10 total flips. Performed 10000 rounds, and     52 were successful, found 36.54% continued the streak.
Looking for a streak of length  8 in   100 total flips. Performed 10000 rounds, and   1634 were successful, found 33.60% continued the streak.
Looking for a streak of length  8 in  1000 total flips. Performed 10000 rounds, and   8365 were successful, found 38.27% continued the streak.
Looking for a streak of length  9 in    10 total flips. Performed 10000 rounds, and     17 were successful, found 47.06% continued the streak.
Looking for a streak of length  9 in   100 total flips. Performed 10000 rounds, and    784 were successful, found 33.04% continued the streak.
Looking for a streak of length  9 in  1000 total flips. Performed 10000 rounds, and   6037 were successful, found 35.80% continued the streak.
Looking for a streak of length 10 in    10 total flips. Performed 10000 rounds, and      0 were successful, found NaN% continued the streak.
Looking for a streak of length 10 in   100 total flips. Performed 10000 rounds, and    381 were successful, found 30.71% continued the streak.
Looking for a streak of length 10 in  1000 total flips. Performed 10000 rounds, and   3615 were successful, found 33.91% continued the streak.

Tuesday, April 30, 2019

Should You Time The Market?
You have 2 investment strategies to choose from.
  1. Dollar-cost averaging (DCA):  You invest $100 (inflation-adjusted) every month for all 40 years.
  2. Buy the Dip: You save $100 (inflation-adjusted) each month and only buy when the market is in a dip.  A “dip” is defined as anytime when the market is not at an all-time high.  But, I am going to make this second strategy even better.  Not only will you buy the dip, but I am going to make you omniscient (i.e. “God”) about when you buy.  You will know exactly when the market is at the absolute bottom between any two all-time highs.  This will ensure that when you do buy the dip, it is always at the lowest possible price.

Making a DIY smartwatch

Friday, March 15, 2019

Everything Smarthome

This is a long, but enjoyable article in broken Russian-English about everything smarthome in 2019.

Wednesday, February 27, 2019

Password strength

Dropbox has a password strength estimator called zxcvbn that I like a lot.  It estimates entropy in your password by looking for dictionary or password list leak matches.  It's long bothered me when sites estimate password strength purely based on complexity.  These sites say a password like Password!1 is much more secure than one like zbuwcramudbpvreorkno (a score of 72% vs 21% respectively).  I discuss this in more detail in my How to be secure online post.

However, a while ago Dropbox changed their algorithm to favor length over resistance to dictionary attacks.  There is some logic in their decision, but I really feel like something is lost by not having the old algorithm.  So, I made a demo comparing the two so you can find passwords both algorithms agree are strong.  At the same time, I finally hooked up this domain I bought a while ago to my github pages site.

Thursday, January 31, 2019


Tuesday, December 25, 2018

How to Be Secure Online: The Blog Post

I've read a lot recently about some new types attacks I wasn't aware of before.  Most of these can be defended against pretty easily, it's just a matter of knowing the threats.  I wanted to summarize some of the things everyone should be doing at this point, but most people aren't.

Use a password manager

At this point, you really should be using a password manager.  You have to assume some of the sites you use will be breached in any given year, and when they are the username and password you use there will be tried on other popular sites.  The only way to be safe is to use different random passwords for every site.  There is no way you can memorize random passwords for every site, even if you limit it to only the sites you actually care about the security of.

However, security isn't the only benefit of a password manager, it is also much more convenient.  You can memorize one really good random password, with no restrictions on maximum length or allowed characters, and then use random passwords on every site.  You'll never have to worry about password complexity restrictions, or being forced to change your password again.  Just generate a new 30 character random password and let the password manager worry about keeping track of it.

I wrote about password managers in more detail here.  If you just want the easiest path, then LastPass will work fine.  I use KeepassXC which is open source and offline.  You have to copy the password file between computers and phones yourself, using something like Dropbox, or the open source Syncthing.

Use a long password

You should only need one or two passwords, if you are using a password manager, so you can make them very strong.  You should make your password very long, and not worry about complexity too much.

I've always been bothered with password strength estimators that score you based on complexity.  A classic example of a bad password estimator is

If I generate a random 20 character password, but one that consists of only lowercase letters like xznmjetjsciqukhspaxv gives that a score of 21% (weak).  A 6 character random password like z&*4uV gets a score of 64% (strong), merely because it has lower case, upper case, digits, and special characters.  Tacking on 2 more characters z&*4uV.9 gets you to 100% (very strong).  While that is an ok password, the 20 character one is much, much stronger, despite being all lower case.  Even if the attacker knew that your password was all lowercases there would still be over 10^28 possibilities.  Trying every possible 6 character password, even with all 95 normal keyboard characters possible, is only about 10^12 possibilities.  Which makes the 20 character password roughly a quadrillion times more secure than the 6 character one.  Even the 8 character one is a trillion times worse than the 20 character one.

Luckily, people are starting to wise up to how useless things like replacing o with 0 are.  NIST has updated password guidelines that are a great summary of what restrictions should be on password systems.  Password estimators like the one above used to be much more common, and even major companies used them.  A long time ago I made my own password estimator, which attempted to replace common dictionary words and then figure out the number of possible combinations, however Dropbox has a way better version of that called zxcvbn, named for the bottom row of letters on a keyboard.  Using zxcvbn as a password would seem random to many estimators, but isn't actually, and attackers were already trying keyboard patterns.

At some point, zxcvbn changed its algorithm for calculating entropy.  I didn't like this change, so I made a page with both the new and old versions of it so you can compare the two.

Don't use SMS for 2 factor authentication

Don't use actual cell phone numbers with a traditional carrier, like Verizon, for 2 factor auth.  It is quite easy, and increasingly common to intercept SMS codes via SIM swapping attacks.  All an attacker needs is your phone number; then they call your carrier and pretend to be you with a new phone and SIM card, and ask for your number to be ported to the new phone.  Then they request a 2 factor auth code and it goes to the phone they have instead of yours.

If you are going to use 2 factor auth, you should use a hardware device like a Yubikey, or an app like Authy.  If the service only supports SMS based 2 factor auth, then use a VOIP number like Google Voice, which can't be easily ported to a new carrier.

The worst part of this, is that using plain SMS for 2 factor auth can make you less secure than no 2 factor auth, because an attacker attempting to social engineer their way into your account will be more believable if they have access to SMS codes being sent to them, versus if there is no 2 factor turned on.  In some cases services allow you to reset your password using only your SMS phone number, so someone who knows your phone number, but not your password, can reset it and get into your account.

Freeze your credit

After the Equifax data breach it's safe to assume that if you have a credit history in the US, that history including SSN and date of birth was leaked.  To open new accounts one typically only needs SSN, DOB and name.  To prove your identity online you are sometimes asked security questions generated from your credit history (things like what bank was your car loan in 2015 with?).  All those things were leaked.

A credit freeze simply adds a random PIN that will be needed to open new accounts, ie, any time someone wants to do a hard pull of your credit with one of the reporting agencies, they will require you to lift the freeze, using the PIN.  Note that you can still use your existing accounts with the freeze in place, it's only opening new accounts that will be blocked.  You can quickly and temporarily remove a freeze (called thawing) within a few minutes.  See here or here for more info on how to freeze your credit.

When freezing your credit, make sure they use the word "Freeze" on the page.  Be careful not to do any sort of credit monitoring or "locking", those are paid services that are less effective than freezes.  They will push those hard, both because they can charge for them, and because people freezing their credit restricts the agencies from doing whatever they want with your info.  Worse still, if the monitoring is with a third party, the will require your SSN and other info to monitor your credit, giving your info to yet another database that will inevitable be leaked at some point.

Friday, November 16, 2018

Invisibly inserting usernames into text with Zero-Width Characters
Zero-width characters are invisible, ‘non-printing’ characters that are not displayed by the majority of applications. F​or exam​ple, I’ve ins​erted 10 ze​ro-width spa​ces in​to thi​s sentence, c​an you tel​​l? (Hint: paste the sentence into Diff Checker to see the locations of the characters!). These characters can be used to ‘fingerprint’ text for certain users.

Sunday, November 4, 2018

The FBI of the National Park Service
Last August, I traveled to Yosemite National Park to meet up with Shott’s colleague, ISB special agent Jeff Sullivan, an affable, self-deprecating, 35-year veteran of the Park Service. Sullivan has played a role in investigating nearly every major crime and mystery that’s taken place in Yosemite over the past quarter-century, which made him the ideal guide for a tour of the shadowy side of America’s fifth most visited national park. See that grassy expanse, dotted with wildflowers? That’s where park visitors discovered the skull of a still-unidentified young woman, a murder claimed by the prolific serial killer Henry Lee Lucas. That lush meadow? Once, someone found a dead bear there, its head neatly severed from its body. (The ISB sent the bear’s remains to the park’s wildlife lab in Oregon, hoping to discover clues about who’d poached it. The lab called back a few weeks later: The poacher you’re looking for is a mountain lion.) 
Sullivan and I drove up to Glacier Point, where he told me about the rockslide in 1996 that killed one and injured at least 11. The dust cloud it kicked up was so massive it blocked out the sun; until Sullivan arrived on the scene, he’d been sure there would be dozens of casualties. Next to us, a bored teenager flung a water bottle into the abyss. Watching it fall seemed to cause Sullivan physical pain. He leaned in close and flashed his badge at the kid. “Don’t throw water bottles,” he said quietly.

Monday, October 22, 2018

How to set up Raspberry Pis without a keyboard, mouse, or monitor

There are plenty of guides out there about how to set up headless Raspberry Pis, but they get out of date quickly, and I do this often enough that I'm constantly searching for up to date ones.  So for my own benefit here's my documentation of the process.

Download Raspbian Lite.  This is the version without the GUI components.

Put your SD card in your computer and use lsblk to identify which drive your SD card is. Be careful, if you use the wrong drive below you will overwrite your main hard drive.

Use dd to copy the date over.  They constantly recommend you use the program Etcher, but I've never had it work successfully.  The command is sudo dd bs=4M if=2018-10-09-raspbian-stretch-lite.img of=/dev/sde conv=fsync status=progress

Your card should have 2 partitions, open the boot partition and add an empty file called ssh to enable ssh, and create a file called wpa_supllicant.conf to configure wifi.  The contents of the file are this:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev


You can either put your actual password in there as the psk, or use the tool wpa_passphrase to convert your password into a hash that will also work.

Put the card in the Pi, boot it up, and it should connect to your network and you should be able to ssh in with username pi and password raspberry.  Note that you need to boot once for it to expand the filesystem.

You should put your public key in ~/.ssh/authorized_keys and turn off password ssh access.  You should also run sudo raspi-config once you ssh in, and update with sudo apt update && sudo apt upgrade

Saturday, October 6, 2018

Blockchain Technology Overview

NIST just published a good overview of blockchain technologies.  Very thorough, yet digestible for non-technical readers.
Blockchains are tamper evident and tamper resistant digital ledgers implemented in a distributed fashion (i.e., without a central repository) and usually without a central authority (i.e., a bank, company, or government). At their basic level, they enable a community of users to record transactions in a shared ledger within that community, such that under normal operation of the blockchain network no transaction can be changed once published. This document provides a high-level technical overview of blockchain technology. The purpose is to help readers understand how blockchain technology works.

GoogleMeetRoulette: Joining random meetings
Let’s see… I generated a meeting, got a Google Meet phone number, and all subsequent phone numbers are also from the same carrier. Let me call and see if I get the Google Meet greeting to confirm. Bingo! For countries like Australia and Spain, Google Meet phone numbers are assigned in batches that are sequential. I can generate a meeting myself and just check the subsequent phone numbers to obtain more Google Meet numbers. You can use them to join/find meetings in the US as the phone numbers from other countries are not specific to meetings in that country, they are global.

10 seconds per call, 3 PINs at a time, 10,000 PINs to try. It would take about 9 hours to cover all PIN combinations making one call at a time. Because Twilio is designed to make calls at scale, we can make hundreds of calls at the same time making the process much faster. The script fires so many calls that the line will be busy sometimes. Not a problem! The script will detect failed calls and simply retry. Actually, Twilio notifies of failed calls immediately using webhooks making the script very efficient handling calls that did not go through.  
I did some benchmarks and on average it takes 25 minutes to try all 10k PINs and find 15 different valid PINs for 15 different meetings for a cost of $16. Not bad!

Saturday, September 29, 2018

Man in the browser attack

Recently I heard of the man in the browser attack and thought it was interesting.  This is malware that is installed in your browser (as an extension for example), and silently waits for you to do a bank transfer.  When you do it can simple change the to account and routing numbers you submit to that of the attacker.  Everything looks fine to you, and wire transfers already take days to process.  Things like strong passwords and 2 factor authentication won't help since you are logging into your real bank's website.

Monday, August 27, 2018

wideNES - Peeking Past the Edge of NES Games
At the end of each frame, the CPU updates the PPU on what has changed. This involves setting new sprite positions, new level data, and —crucially for wideNES— new viewport offsets. Since wideNES runs in an emulator, it’s really easy to track the values written to the PPUSCROLL register, which means it’s incredibly easy to calculate how much of the screen has scrolled between any two frames!

Hmm, what would happen if instead of painting each new frame directly over the old frame, new frames are instead painted overlapping the previous frame, but offset by the current screen scroll? Well, over time, more and more of the level would be left on-screen, gradually building up a complete picture of the level!

Friday, August 24, 2018

How I recorded user behaviour on my competitor’s websites
I spoofed the back button in Chrome and sent people to my version of search results and competitor websites where I recorded everything with Lucky Orange.

Friday, May 25, 2018

How Ikea took over the world
One way Ikea researchers get around this is by taking a firsthand look themselves. The company frequently does home visits and—in a practice that blends research with reality TV—will even send an anthropologist to live in a volunteer’s abode. Ikea recently put up cameras in people’s homes in Stockholm, Milan, New York, and Shenzhen, China, to better understand how people use their sofas. What did they learn? “They do all kinds of things except sitting and watching TV,” Ydholm says. The Ikea sleuths found that in Shenzhen, most of the subjects sat on the floor using the sofas as a backrest. “I can tell you seriously we for sure have not designed our sofas according to people sitting on the floor and using a sofa like that,” says Ydholm.

Monday, March 12, 2018

Smart homes and vegetable peelers
Many of the things that get a connection or become 'smart' in some way will seem silly to us, just as many things that got 'electrified' would seem silly to our grandparents - tell them that you have a button to adjust the mirrors on your car, or a machine to chop vegetables, and they'd think you were soft in the head, but that's how the deployment of the technology happened, and how it will happen again. The technology will be there, and will become very very cheap, so it will slide unnoticed into our lives. On the other hand, many things that people did think might get electrified did not, and many of the ideas that did work were not adopted in a uniform way. Most people in the UK have an electric kettle, but that's not true in the USA, and most people in Japan have a rice cooker, but this in turn isn't true in the UK. Anyone who's baked a few times has bought an electric whisk for $20, but not many people use electric carving knives.

Friday, February 16, 2018

The “hydrogen economy” may be a thing after all.
The first product, scheduled to debut in April, is the key to everything else.
It’s called Internal Combustion Assistance (ICA), a modification to internal combustion engines that enables them to substantially increase their fuel efficiency and reduce their air pollution. It does this by adding tiny amounts of gaseous hydrogen and oxygen to the fuel just before it is combusted in the engine’s cylinders. The HHO mix lends intensity to the combustion, allowing the fuel to burn more completely, generating more oomph and less pollution.
The ICA system can technically work on any internal combustion engine, but to begin with, HyTech is targeting the dirtiest engines with the fastest return on investment, namely diesel engines — in vehicles like trucks, delivery vans, buses, and forklifts, but also big, stationary diesel generators, which still provide backup (and even primary) power by the millions across the world.

Let's Learn About Waveforms

Monday, January 29, 2018

Password Management

I've long maintained that the only sites that really need strong passwords are emails (because they let you reset other passwords) and financial sites.  I've memorized long random passwords for those sites, and I have a few similar passwords I use for the rest of things.  I've never been too concerned about sharing passwords between other sites, because I literally don't care about the security of those accounts.

That being said, sites are increasingly instituting arbitrary restrictions that are intended to make things more secure.  This means I need variations of my common passwords for every permutation of rules, and then variations of those for when I'm required to change them.  Having to try all these permutations has finally made me break down and start using a password manager.

It's probably no surprise I didn't just go with LastPass, and not just because of my general aversion to the most popular choices, but as I've heard the company they are owned by is shady.

Password Manager

There are a lot of password managers, but if you're looking for open source, and managing your own password file, the clear choice is KeePass.  However, as is an open source tradition, you can't just go with KeePass; you have to follow the forks, to find the version that is currently up to date and being maintained.  That version is KeePassXC.

If you go with KeePassXC you'll have a client on every device you want to use it with.  Then you'll have a password file, which is the encrypted file holding all your passwords.  In theory if your master password for that file is long and secure you won't need to worry about keeping that file too safe (don't post it publicly).  I'd recommend getting to at least centuries on the 10k/second tier of zxcvbn.

You can also use a keyfile, which is a random file you'll need in addition to a master password to decrypt your password file.  This adds some security, but keep in mind that if someone gains access to a device with your password file, they also probably gain access to the keyfile.  It mainly helps if you are worried about your password file getting intercepted during syncing between devices (you wouldn't sync the keyfile, you'd move it manually to new devices).

Syncing the Password File

This felt like it was going to be the hardest part, but it turned out to be the easiest.  Certainly, the biggest convenience of LastPass is that someone else manages the password file for you.  A lot of people use Dropbox to sync the KeePass file, and I was ok with this (as the file is encrypted so you aren't really trusting Dropbox with anything), but I hate the idea of installing Dropbox's bloated, always running, client on every device.

Luckily I found Syncthing.  Which is essentially an open source, bit torrent based, version of Dropbox.  You install it on all your machines and then point it to the folder you want to share and it keeps it synced.  My biggest issue was having to enable discovery on every device so that they would share the list of devices they are sharing with too.  This makes sense to have turned off if you were sharing with other people, but if you're only using it in a closed personal ecosystem it's much easier to have it enabled.

I was slightly worried about the password file becoming out of sync, getting written to by two different computers and getting corrupted.  But my mild stress tests have been unable to make this happen.  I've been using this set up for half a year now without issue, so I'm comfortable recommending it.  That being said, Syncthing does allow you to maintain history files (where it keeps the last few versions of the file every time it overwrites it), and I still have that enabled on my PC.

Browser Integration

KeePassXC uses a protocol called KeePassHTTP to share passwords externally.  This basically just sets up a server and allows http requests for your passwords.  This is risky because there could be external requests.  KeePassXC only allows localhost requests, which should mitigate that risk.  If you're still worried you can disable that and use autotyping where you place the cursor in any text field and the it types the password in that field.

Just searching for "KeePassXC Firefox" or Chrome shows the extensions for either.  I've been happy with both of those, although they do feel like the weakest link.

On Android the app Keepass2Android works well.  If you search for the site in the app it then gives you another keyboard to choose from which only has two buttons "User" and "Pass".  Pressing those fills in that info for the site you have selected.

The closest thing to a problem on the phone is that it takes a few seconds to unlock the file.  This is important though, it should take at least half a second to unlock your password file on a fast PC.  If you make it faster to open, it'll be easier to brute force.

Thursday, December 28, 2017

Create fake videos of famous people saying anything you want in real time.

Google has software that can create dynamic audio indistinguishable from a real person (scroll to "Tacotron 2 or Human?" at the bottom):

Combine with this which lets you to put your facial movements on a real person:

Wednesday, December 20, 2017

Google Map's Moat
Annechino and Cheng spent months researching one city. But not only did Google capture all of their commerical corridors (and several more), it somehow came up with them for thousands of cities across the world. (Even my tiny hometown got a few.)

Sunday, October 29, 2017

Designing a Hammock Stand

2017 note: This is a post I wrote in 2013 about a hammock stand I never built.  My plan was to build it first so I could add finished pictures, but I think I'm about ready to give up on my prospects of ever actually building this.

Every night I sleep on, what is essentially, a pile of garbage.  A while back, someone started a thread on reddit about sleeping in a hammock every night.  He had nothing but praise for hammocks and some research showed the internet generally agreed that hammocks were an excellent bed replacement.

I decided I would give it a try.  As I backpack too, I bought an ENO Doublenest that can be used for camping.  Although I have no real plans to do that.


Before I could plan a stand for nightly, indoor, hammock use, I had to know what kind of forces it would have to handle.

The hammock's max weight rating is 400 lbs, and since it is probably better for the hammock to fail before the stand I used that as my load.

Each end of the hammock has to support half the load, however, this is not just 200 lbs.  To find out why, and see what the actual tension is we will break the force vector into its x and y components

We don't know the x force, and are trying to find the resultant force.  We do, however, know the angle and the y force.  The y force must be equal to just half the weight supported by the hammock.  This assumes the weight won't get shifted too much to one end.

The ideal hang angle is widely reported to be 30 degrees down from a horizontal.  A preliminary test of mine showed I liked it closer to 45.  Shallower angles increase the stresses, so I planned for 30 degrees.

The forces form a right triangle with all known angles and one known side.  It's a 30-60-90 triangle and the vertical leg is 200 lbs.  This means the horizontal force is 347 lbs and the resultant force on the line is 400 lbs.  To be clear, this means that for a 30 degree hang, each line must support the full weight in the hammock.  If we let the hammock sag more to 45 degrees it reduces the tension on the line to only 283 lbs.

Hanging Possibilities

The guy in the reddit thread said he simply hung his from eye bolts in the wall studs.  I don't have wall studs in the basement, and I wouldn't trust them if I did.  I considered hanging from the 2x8 ceiling joists.  The problem was the height would mean I would need a huge span between the two hang points.  Also, I wouldn't recommend anyone hang perpendicular to the direction of the joists, as this will cause deflection.  It might not seem like it would be a big deal, but it is generally a bad idea to introduce new stresses in directions that structural members were never designed to handle, particularly when they are holding up your house.

After some debate I decided on building a stand.  Since most the force is in the horizontal direction I thought about just hanging from a 4x4 post that I would elevate off the ground with some sort of stands on the ends.  I didn't like this idea since it would be annoying to have the post above me while sleeping, and even more annoying when it failed and crashed down on my face.

In order to deal with the high horizontal load, whatever the hammock is actually hanging from would have to be angled out.  This leads to the classic hammock stand shape of angled arms.

Will 2x4s Work?

I decided to angle my arms out at 60 degrees above the horizontal.  This meant that the hammock would hang between 30 and 15 degrees above the arm (for 30 to 45 degrees below horizontal).  To calculate the stresses in the arm I rotated the axes such that the arm was now vertical.  I then calculated the x and y force vectors in this new rotated orientation.

Drawing out the forces shows the 30 degree hang produces the same force triangle as before, just flipped.  There will be 200 lbs of force perpendicular to the arm, and 347 lbs of force parallel to it.  For the 45 degree hang it shifts to 274 lbs parallel to the arm and just 74 lbs perpendicular to it.  You can see what a significant factor the hang angle is to the forces involved.  If I were actually planning on hanging at 30 degrees I'd probably adjust the arms out further to shift more of the torque to compression.

We now have two forces which means we can see what types of load they produce on the arms.  The parallel force results in a axial compressive load, ie, like a column.  This handy calculator tells me that a 2x4 can support 1000-1500 lbs of compression over 5 unbraced feet, depending on grade.  So axial load shouldn't be a limiting factor.

The perpendicular force will produce moment (torque).  The amount will depend on the arm length, which is not yet known.  An estimate of 4.5 feet gives us 340 ft lbs for the 45 degree hang, and 900 ft lbs for the 30 degree hang.  The above calculator's brother tells me that a 2x4 should be able to handle about 375 ft lbs of bending.  This roughly matches the calculations I did on paper as a sanity check.

So, we can see that for a 30 degree hang a 2x4 wouldn't be enough.  Keep in mind we started with a load of 400 lbs in the hammock, and I'd guess that calculator has a safety margin built in, so I'd guess that a 2x4 would hold, at least for a while.

My Design

I began my design with this simple design.  Reading through the comments and several other sites I changed the design significantly.  A goal of my design was ease of construction with a total lack of tools and experience (which admittedly might impair my ability to judge what is easy to build).

After some concern about the torque in the joint I decided to create something like a half lap joint.  I will layer two 2x4s to make a quasi 4x4 as my horizontal base.  First, however, I will make a 60 degree cut through one of the 2x4s at about 2 feet from the center.  I will cut the other 2x4 at 2 feet from the center in the other direction.  Thus, both 2x4s will have a 60 degree cut, 2 feet from the center, but on opposite sides of the center.  I will then gap the cut enough to fit the 2x4 for the arm in there, and cut the bottom of the arm at 60 degrees to make it flush against the floor.  This means the two arms won't be exactly in line, but rather slightly offset.  Looking at the stand head on, the left edge of one arm will be aligned with the right edge of the other.

The two 2x4s along the base will be wood glued and screwed to form a solid 4x4 like piece.  At the end of the base 4 foot long 2x4s will be screwed into the base, forming an I shape.  They will provide stability.  At this point the design looks similar to the starting design, albeit with major, if subtle, differences.  The last change would be side bracing similar to what people in the comments added.  Each side brace would be about 4 feet long and set up at a 60 degree above horizontal angle.  For no reason I decided to put each brace on opposite sides of the ground 2x4 they connect to.  That changes their length by a bit.  It took me much longer than I care to admit to calculate what that difference would be, finally resorting to just using CAD.

Here are some preliminary CAD drawings I did.  I still plan on testing the hang length a bit more before these are final.  Since I'm well aware that no one could be expected to visualize what I described, and that these drawings don't help much, I won't publish this until I'm ready to follow this post up with a construction post with actual pictures.

In lieu of finished shots I did this render in Tinkercad:

Friday, September 22, 2017

10 iconic logos. 156 Americans. 80 hours of drawing from memory.

A High-End Mover Dishes on Truckstop Hierarchy, Rich People, and Moby Dick
Since I now work for a boutique van line doing high-end executive moves, all of my work is what we call pack and load. That means I’m responsible for the job from beginning to end. My crew and I will pack every carton and load every piece. On a full-service pack and load, the shipper will do nothing. I had one last summer that was more or less typical: The shipper was a mining executive moving from Connecticut to Vancouver. I showed up in the morning with my crew of five veteran movers; the shipper said hello, finished his coffee, loaded his family into a limousine, and left for the airport. My crew then washed the breakfast dishes and spent the next seventeen hours packing everything in the house into cartons and loading the truck. At destination, another crew unpacked all the cartons and placed everything where the shipper wanted it, including dishes and stemware back into the breakfront. We even made the beds. We’re paid to do all this, of course, and this guy’s move cost his company $60,000. That move filled up my entire trailer and included his car. It was all I could do to fit the whole load on without leaving anything behind, but I managed it. I do remember having to put a stack of pads and a couple of dollies in my sleeper, though.

Wednesday, August 30, 2017

A history of branch prediction from 1500000 BC to 1995
One way you might design a CPU is to have the CPU do all of the work for one instruction, then move on to the next instruction, do all of the work for the next instruction, and so on. There’s nothing wrong with this; a lot of older CPUs did this, and some modern very low-cost CPUs still do this. But if you want to make a faster CPU, you might make a CPU that works like an assembly line. That is, you break the CPU up into two parts, so that half the CPU can do the “front half” of the work for an instruction while half the CPU works on the “back half” of the work for an instruction, like an assembly line. This is typically called a pipelined CPU.

Sunday, July 9, 2017

Cats Lasers Robots


The Raspberry Pi has really come along nicely.  This year for Pi Day they released an version of the $5 Pi Zero, which has wifi and costs $10.  That's $10 for a full computer with wifi, and bluetooth, which is pretty amazing (you do have to find or buy a 8GB microSD card and a micro USB power supply, so actual costs are closer to $25, but still).

I bought one without any real purpose in mind.  Around the same time my girlfriend bought a cat toy call the "Bolt".  It's a laser which reflects off a mirror and makes a large arc on the floor, randomly changing directions.  There's a single button on the back to turn it on/off.

I figured the button was just shorting something to turn it off and on, and I could replicate that with a Pi to enable it to be web controlled.

Before I began, I had some requirements in mind:
  • The finished product had to be fairly well polished.  It had to look, at least at first glance, like a consumer product.  
  • It had to just work when plugged in, I could spend as much time as I needed hardcoding wifi passwords ahead of time, but the end result had to be plugging it in and it working.  
  • The normal button the back of the toy had to work the same as always.  
  • The interface had to be relatively simple to use, I was ok with a page that could be bookmarked.


The toy took 4 AA batteries which means it used around 5V and I could probably power it from the Pi as well.  The Pi uses 5.25 V, and while you can't power things from the GPIO pins, there is a 5.25 V pin that is a straight connection to your power supply.  The Pi power supplies are generally 1 or 2 amps, and the Pi Zero needs like 200 mA, so I figured I'd be fine on power.

So I got the toy and I cracked it open to see what was what.  It opened pretty well considering there were no screws.  The wiring was pretty simple.  Two wires supplying power from batteries, and then two wires connecting the button.

The first step was seeing if 5.25 V would even work.  AA batteries are nominally 1.5 V, which means it would be 6 V.  However, they drop off in voltage quickly, and rechargeable batteries are 1.2 V which would give 4.8 V, so it had to be fairly robust.  I hooked up a power supply, and set it to 5.25 V and confirmed everything worked.  Then I measured the voltage across the push button and confirmed it was just 5.25 V. 

The next step was cutting out the battery compartment, and confirming that 3.2 V from the GPIO pins would turn it on.  I measured the current draw of the toy at about 200 - 400 mA, which would be easily handled by my power supply.  Finally, I confirmed that the actual 5.25 V pin on the Pi could power the toy.  At this point I figured the hardware was settled, I just had to figure out how to send a command to a Pi.


This is where I ran into some troubles.  While I knew a lot of ways I could do this in theory, I didn't want to have to mess with routers and port forwarding.  My first plan was to use Twilio and use SMS to control it.  However, looking into it, Twilio just converts SMS into API calls, I'd still need an API, and some way for the Pi to connect to it.

The low tech way of doing that is to just poll the API constantly.  That works, but it lacks elegance, and I'm all about elegance.

It turns out that Rails 5 supports websockets, which is the ideal way of doing this.  Websockets are just an extension to http.  Essentially websockets start as a http request, and the server just leaves the connection open.  There's more to it than that, but it's really just a standard around leaving connections open so that servers can send messages to clients without the client having to request it each time.

Websockets API

I got to work on making a Rails API, which was pretty straight forward.  The websockets stuff was also pretty easy, as Rails tends to be.  However, when it came time to make a client, I couldn't get the format of the requests right.  I was attempting to use Python, and whatever their websockets library is, but I decided to look for implementations that were designed with the Rails websockets server in mind.

I ended up using this project, which is designed to work with Rails.  Once I switched to that, the rest of the API work went quite fast.

Websockets Client

Next I made the Pi client that would listen for websocket events and turn on the cat laser.  The basic idea was simple: I found a Ruby gem to do GPIO stuff, and set it to drive my pin high for half a second.  I tested it with the hardware and everything worked (amazingly).  The hard part came in making the client robust.  This thing had to be very user friendly.  It had to just work.

The gem I was using had some hooks for unsubscribed, but I quickly learned they weren't reliable.  Further investigation revealed that there was a ping that came through every 3 seconds.  My plan was to record that and attempt to reconnect when it got old.  However, I couldn't get that gem to reconnect successfully.  My final plan was just to write the ping timestamps to a file, and then have the script end when they got old.  A separate script would check for ping age and restart the main script when it saw them old.  I set up a ramdisk for the ping file so it wouldn't kill my SD card.

This felt pretty hacky, but worked very well.  Every method of artificial connection problems I could simulate were handled by this.  It could take up to a minute to reconnect, but that was fine, and was mainly due to me running this as a cron job.  If reconnecting faster were really an issue I could do it in a loop.

Hardware, part II

With that I had a pretty solid setup.  I began to plan on how I would wire this all up.  While the hardware was simple, I was most worried about messing something up there.  It was around this time that I realized there was a flaw in my hardware plans.  I was planning on hooking a GPIO pin directly up to the low side of the push button.  I would raise it to 3.2 V and that would turn on the toy.  You could also press the button and it would raise it to 5.25 V as it normally would.  This let you use the normal button the same as always.  However, the button would also short 5.25 V to the GPIO pins, which would kill the Pi (or at least the pin).  My first thought was to use a diode, which basically act as a one way valve for voltage, but they also drop the voltage across them, and it was already lower than it should be at 3.2 V.  My tests showed the diode was unreliable.

The failed setup

My next plan was a transistor.  Transistors are both sophisticated and simple, but for my purposes I could treat them as a voltage controlled switch.  I used an NPN transistor I had laying around and connected the collector to the high side of the switch, and the emitter to the low side.  I could then supply 3.2 V to the base to send 5.25 V to the low side of the switch and turn the toy on.  Pressing the button normally would short the emitter and collector, which would be fine.  I tested this set up and it seemed to work, although it was getting difficult to test all these connections with the toy physically moving around when it turned on, and the Pi having no headers to plug stuff into securely.

The winning setup

I used this as an excuse to buy something I had my eyes on for quite some time.  This fancy third hands tool.  You can get these things for like $5, but this one has a reputation for being very versatile and well thought out.  Plus they included a bag of Swedish Fish in the box, which made me happier than anything else in recent memory.

At this point I had three wires.  One I had soldered to the low side of the switch, and then the 5.25 V and ground supplies coming from the Pi.  I shrink tubed the solder joints to protect them (after one broke).  I began thinking about how the Pi would fit inside.  The Pi zero is very small, and there was a good amount of empty space inside the toy, particularly where the batteries had gone, so fitting it wasn't a problem.  However, I wanted it to be secured in there so I wouldn't have to worry about it coming lose and putting stress on the wires.  There were four screw posts where the battery compartment had been attached.  I decided this would work perfect to attach one of the corners of the Pi.  I spent a while going over the possibilities.  There were a lot of ways the Pi almost fit, but there seemed to be one choice that was the best out of the ways it did fit.

I soldered the wires to the pins on the Pi, and I attempted to drill a hole for the cord, only to discover the plastic was having none of that.  I resorted to using pliers to cut and twist the plastic apart.  This actually worked far better than I would have expected, and the end result was pretty presentable looking.

I plugged it all in and tested it with the API hosted on Heroku.  Amazingly it worked.  I tested rebooting the server and killing the wifi and other permutations, and the client consistently reconnected.


The API worked well, although it was a bit clunky to use, having to bookmark a page with the basic auth username and password built in.  This gave a warning on most browsers that you had to click through.  Ultimately a legitimate front end would solve this, but in the short term I decided to bring in yet another technology

I was aware that Alexa had an API to perform custom actions.  Setting it up took a few hours, mainly due to how cryptic Amazon is about everything they do.

First you need to create a lambda function.  Lambda functions are just short scripts you write in Javascript or Python and Amazon runs them when you hit some endpoint.  They're pretty straight forward.  I used Python 2.7, and set up a "role" (Amazon's permissions model) with whatever basic preset was available.  I then set the trigger to be "Alexa Skills Kit".  My code was just the color sample code, with all my code in the get_welcome_response method.  That method gets called when the Alexa runs the lambda and all I had it do was hit my API.

At this point you get an ARN which is what you need the Alexa to call to run your lambda.  The second half was much more confusing.  First, for some reason all the Alexa stuff is not in AWS, but rather the "Developer Console".  Once I found that I created a new Alexa skill.  There is a ton of configuration for the skills, but for the most part I either left it as defaults, or googled values to enter for things like "Intents".  The only real configuration I had to do was to enter my ARN as the endpoint, and enter what I wanted to say to turn it on as "Invocation Name".  Once I got to the testing step I enabled that and it worked.  I didn't have to fill out Publishing or Privacy details.

While the skill worked for me, I wanted to make it available to other users, while not actually publishing it.  After hunting around I discovered you can invite users to be developers in Settings > User Permissions.  They then have to accept, and go into the developer console and enable testing in the skill.  It will then show up as a custom skill in the Alexa app.

With this, the command "Alexa, laser" would turn it on/off, and it worked pretty well.  The only hiccups have been in Alexa failing to understand what is being said.

How's it work?

 This setup has been running for 3 months now, and has been amazingly robust.  There has been exactly one case of the API and client not working, and that was caused by the Pi losing its wifi connection for some reason and then failing to reconnect.  Unplugging the Pi fixed it.  I then set up another script to run on the Pi to check the last ping timestamp and restart the Pi if that is a few minutes old.

The code for the controller and client are on Github:

Saturday, June 3, 2017

Network Protocols
TCP has no special "I lost a packet!" message. Instead, ACKs are cleverly reused to indicate loss. Any out-of-order packet causes the receiver to re-ACK the last "good" packet – the last one in the correct order. In effect, the receiver is saying "I received packet 5, which I'm ACKing. I also received something after that, but I know it wasn't packet 6 because it didn't match the next sequence number in packet 5."

If two packets simply got switched in transit, this will result in a single extra ACK and everything will continue normally after the out-of-order packet is received. But if the packet was truly lost, unexpected packets will continue to arrive and the receiver will continue to send duplicate ACKs of the last good packet. This can result in hundreds of duplicate ACKs.

When the sender sees three duplicate ACKs in a row, it assumes that the following packet was lost and retransmits it. This is called TCP fast retransmit because it's faster than the older, timeout-based approach. It's interesting to note that the protocol itself doesn't have any explicit way to say "please retransmit this immediately!" Instead, multiple ACKs arising naturally from the protocol serve as the trigger.

Tuesday, May 16, 2017

MP3s as a litmus test for good journalism

The MP3 format was invented by a German group in the early 90s.  They patented it, and licensed it out to companies.  This is the reason many open source programs force you to download MP3 libraries separately.

The last patents for mp3 expire this year (2017).  Now anyone can use it without having to worry about licenses.  The group that created it announced they would stop licensing it (since they can't) and suggested people move to AAC (since they still own patents on that).

The result is news organizations running stories with headlines like "MP3 is Dead".  This presents and interesting look into which sources are reliable sources for tech news, and which use hyperbolic headlines for the sake of clicks. 

I went to Google News and searched for recent articles that mentioned 'MP3'.  Some of these were pretty obvious, but some were surprising.  To be fair, some are technically correct, in saying the creator declared it dead, vs saying it actually is dead, but merely parroting a press release is still going under the 'Bad' category.  The BBC was close, but I put it in good because it didn't feel clickbaity to me, feel free to disagree.

Finally, I won't pretend like this single example is some end all test for who you should and shouldn't trust, it's just and interesting source of some empirical data.


NPR: The MP3 Is Officially Dead, According To Its Creators
The Atlantic: The End of the MP3
Gizmodo: Developers of the MP3 Have Officially Killed It
The Register: MP3 'died' and nobody noticed
Quartz: Say goodbye to the iconic MP3
CNBC: The MP3 is dead, say creators after terminating licensing
The Telegraph: Creators of the MP3 declare it dead
Tech Radar: RIP MP3 - the sound file that changed the world is declared dead


Washington Post: Your MP3s are going to be just fine
Mashable: The MP3 isn't dead yet, but it's now on its last digital legs
Vice: The MP3 Is Not Dead
CNET: MP3 isn't dead, it's just sleeping
BBC: It might be time to say goodbye to the MP3 - so let's look back at its life

Friday, May 12, 2017

Are Pop Lyrics Getting More Repetitive?

This is some good data, but the presentation is very interesting as well.

Tuesday, April 25, 2017

Github has all the best cat stories
In broad daylight, we could see why this was street cat utopia. What used to be a deli or some other food store collapsed in what looks like the 80s. A tree had grown through the inside where the roof had collapsed, a branch somehow punching through brick wall and completely enveloping a piece of old metal shelving. There was no way into this place past the first few steps. The roof was collapsed with a capital C. You could see through the busted rafters towards the middle of the (what was now) one big room of the first floor, and to the street cats that were lazily napping in the sun, protected by their fortress.

Tuesday, March 14, 2017


I've been reading through these replies about "MediTech" trying to determine if it's some sort of elaborate inside joke I'm not picking up on.
There is a company called MediTech in Massachusetts that uses a derivative language of MUMPS called Magic. I know several programmers that have worked there. There are thousands of engineers writing in this language as we speak.
From what I can remember:

-Only global variables

-Variables must only be capital letters, maximum length 6. If you run out of variables, you must cleverly use them in a routine and set them back to what they are. This means you can't use a name like myVar - you use AAAFD, ZBVCXZ, etc.

-System functions are usually things like ., >, ', ], so code looks like .'AAAF]{\;:..

-Meditech writes all of their own languages, databases, operating systems, tools, etc. You can only write in a non-Meditech language if you get approval from a multi-tiered architectural design board, which barely ever happens

-The founder hated C with undying passion. No one is ever allowed to use C

-All programming hires go through a 6 to 12 month training process to learn the tools, languages, and systems. As they almost exclusively hire non-CS majors, such as math and physics majors, they don't typically have a programming background and don't realize how bizarre the MediTech stack is

Monday, March 6, 2017

A Good Overview of How Trump Operates

I try not to post a lot of political or topical stuff here, but this is a very good overview of Trump and how he operates.  It goes into a lot more background and detail than just the current Russia story.
Whenever he is under fire for something in a sustained way, he makes a shocking claim or provocative declaration about something else to change the subject. He is a master practitioner at the politics of distraction. These five examples might jog your memory:
  • After struggling during the first GOP primary debate to explain his disparaging comments about women, he attacked Megyn Kelly. “There was … blood coming out of her wherever,” he said, ensuring that the media focused on the new Trump-Kelly “feud.”
  • In November, the morning after agreeing to settle a fraud lawsuit against Trump University for $25 million, he demanded that the cast of “Hamilton” apologize to Mike Pence.
  • Perturbed when critics pointed out that he lost the popular vote, he claimed that 3 million to 5 million people voted illegally.

Saturday, February 11, 2017

Top mentioned books on

We analysed more than 40 000 000 questions and answers on to bring you the top of most mentioned books (5720 in total)

How we did it:
  • We got database dump of all user-contributed content on the Stack Exchange network (can be downloaded here)
  • Extracted questions and answers made on stackoverflow
  • Found all links and counted it
  • Created tag-based search for your convenience
  • Brought it to you

Saturday, January 28, 2017

Overjustification effect
The overjustification effect occurs when an expected external incentive such as money or prizes decreases a person's intrinsic motivation to perform a task. The overall effect of offering a reward for a previously unrewarded activity is a shift to extrinsic motivation and the undermining of pre-existing intrinsic motivation. Once rewards are no longer offered, interest in the activity is lost; prior intrinsic motivation does not return, and extrinsic rewards must be continuously offered as motivation to sustain the activity.

Sunday, January 15, 2017

The Line of Death
The Metro/Immersive/Modern mode of Internet Explorer in Windows 8 suffered from the same problem; because it was designed with a philosophy of “content over chrome”, there were no reliable trustworthy pixels. I begged for a persistent trustbadge to adorn the bottom-right of the screen (showing a security origin and a lock) but was overruled. One enterprising security tester in Windows made a visually-perfect spoofing site of Paypal, where even the user gestures that displayed the ephemeral browser UI were intercepted and fake indicators were shown. It was terrifying stuff, mitigated only by the hope that no one would use the new mode.