Sunday, July 9, 2017

Cats Lasers Robots


Intro

The Raspberry Pi has really come along nicely.  This year for Pi Day they released an version of the $5 Pi Zero, which has wifi and costs $10.  That's $10 for a full computer with wifi, and bluetooth, which is pretty amazing (you do have to find or buy a 8GB microSD card and a micro USB power supply, so actual costs are closer to $25, but still).

I bought one without any real purpose in mind.  Around the same time my girlfriend bought a cat toy call the "Bolt".  It's a laser which reflects off a mirror and makes a large arc on the floor, randomly changing directions.  There's a single button on the back to turn it on/off.

I figured the button was just shorting something to turn it off and on, and I could replicate that with a Pi to enable it to be web controlled.

Before I began, I had some requirements in mind:
  • The finished product had to be fairly well polished.  It had to look, at least at first glance, like a consumer product.  
  • It had to just work when plugged in, I could spend as much time as I needed hardcoding wifi passwords ahead of time, but the end result had to be plugging it in and it working.  
  • The normal button the back of the toy had to work the same as always.  
  • The interface had to be relatively simple to use, I was ok with a page that could be bookmarked.

Hardware

The toy took 4 AA batteries which means it used around 5V and I could probably power it from the Pi as well.  The Pi uses 5.25 V, and while you can't power things from the GPIO pins, there is a 5.25 V pin that is a straight connection to your power supply.  The Pi power supplies are generally 1 or 2 amps, and the Pi Zero needs like 200 mA, so I figured I'd be fine on power.

So I got the toy and I cracked it open to see what was what.  It opened pretty well considering there were no screws.  The wiring was pretty simple.  Two wires supplying power from batteries, and then two wires connecting the button.



The first step was seeing if 5.25 V would even work.  AA batteries are nominally 1.5 V, which means it would be 6 V.  However, they drop off in voltage quickly, and rechargeable batteries are 1.2 V which would give 4.8 V, so it had to be fairly robust.  I hooked up a power supply, and set it to 5.25 V and confirmed everything worked.  Then I measured the voltage across the push button and confirmed it was just 5.25 V. 

The next step was cutting out the battery compartment, and confirming that 3.2 V from the GPIO pins would turn it on.  I measured the current draw of the toy at about 200 - 400 mA, which would be easily handled by my power supply.  Finally, I confirmed that the actual 5.25 V pin on the Pi could power the toy.  At this point I figured the hardware was settled, I just had to figure out how to send a command to a Pi.

Software

This is where I ran into some troubles.  While I knew a lot of ways I could do this in theory, I didn't want to have to mess with routers and port forwarding.  My first plan was to use Twilio and use SMS to control it.  However, looking into it, Twilio just converts SMS into API calls, I'd still need an API, and some way for the Pi to connect to it.

The low tech way of doing that is to just poll the API constantly.  That works, but it lacks elegance, and I'm all about elegance.

It turns out that Rails 5 supports websockets, which is the ideal way of doing this.  Websockets are just an extension to http.  Essentially websockets start as a http request, and the server just leaves the connection open.  There's more to it than that, but it's really just a standard around leaving connections open so that servers can send messages to clients without the client having to request it each time.

Websockets API

I got to work on making a Rails API, which was pretty straight forward.  The websockets stuff was also pretty easy, as Rails tends to be.  However, when it came time to make a client, I couldn't get the format of the requests right.  I was attempting to use Python, and whatever their websockets library is, but I decided to look for implementations that were designed with the Rails websockets server in mind.

I ended up using this project, which is designed to work with Rails.  Once I switched to that, the rest of the API work went quite fast.

Websockets Client

Next I made the Pi client that would listen for websocket events and turn on the cat laser.  The basic idea was simple: I found a Ruby gem to do GPIO stuff, and set it to drive my pin high for half a second.  I tested it with the hardware and everything worked (amazingly).  The hard part came in making the client robust.  This thing had to be very user friendly.  It had to just work.

The gem I was using had some hooks for unsubscribed, but I quickly learned they weren't reliable.  Further investigation revealed that there was a ping that came through every 3 seconds.  My plan was to record that and attempt to reconnect when it got old.  However, I couldn't get that gem to reconnect successfully.  My final plan was just to write the ping timestamps to a file, and then have the script end when they got old.  A separate script would check for ping age and restart the main script when it saw them old.  I set up a ramdisk for the ping file so it wouldn't kill my SD card.

This felt pretty hacky, but worked very well.  Every method of artificial connection problems I could simulate were handled by this.  It could take up to a minute to reconnect, but that was fine, and was mainly due to me running this as a cron job.  If reconnecting faster were really an issue I could do it in a loop.


Hardware, part II

With that I had a pretty solid setup.  I began to plan on how I would wire this all up.  While the hardware was simple, I was most worried about messing something up there.  It was around this time that I realized there was a flaw in my hardware plans.  I was planning on hooking a GPIO pin directly up to the low side of the push button.  I would raise it to 3.2 V and that would turn on the toy.  You could also press the button and it would raise it to 5.25 V as it normally would.  This let you use the normal button the same as always.  However, the button would also short 5.25 V to the GPIO pins, which would kill the Pi (or at least the pin).  My first thought was to use a diode, which basically act as a one way valve for voltage, but they also drop the voltage across them, and it was already lower than it should be at 3.2 V.  My tests showed the diode was unreliable.

The failed setup


My next plan was a transistor.  Transistors are both sophisticated and simple, but for my purposes I could treat them as a voltage controlled switch.  I used an NPN transistor I had laying around and connected the collector to the high side of the switch, and the emitter to the low side.  I could then supply 3.2 V to the base to send 5.25 V to the low side of the switch and turn the toy on.  Pressing the button normally would short the emitter and collector, which would be fine.  I tested this set up and it seemed to work, although it was getting difficult to test all these connections with the toy physically moving around when it turned on, and the Pi having no headers to plug stuff into securely.

The winning setup

I used this as an excuse to buy something I had my eyes on for quite some time.  This fancy third hands tool.  You can get these things for like $5, but this one has a reputation for being very versatile and well thought out.  Plus they included a bag of Swedish Fish in the box, which made me happier than anything else in recent memory.




At this point I had three wires.  One I had soldered to the low side of the switch, and then the 5.25 V and ground supplies coming from the Pi.  I shrink tubed the solder joints to protect them (after one broke).  I began thinking about how the Pi would fit inside.  The Pi zero is very small, and there was a good amount of empty space inside the toy, particularly where the batteries had gone, so fitting it wasn't a problem.  However, I wanted it to be secured in there so I wouldn't have to worry about it coming lose and putting stress on the wires.  There were four screw posts where the battery compartment had been attached.  I decided this would work perfect to attach one of the corners of the Pi.  I spent a while going over the possibilities.  There were a lot of ways the Pi almost fit, but there seemed to be one choice that was the best out of the ways it did fit.



I soldered the wires to the pins on the Pi, and I attempted to drill a hole for the cord, only to discover the plastic was having none of that.  I resorted to using pliers to cut and twist the plastic apart.  This actually worked far better than I would have expected, and the end result was pretty presentable looking.

I plugged it all in and tested it with the API hosted on Heroku.  Amazingly it worked.  I tested rebooting the server and killing the wifi and other permutations, and the client consistently reconnected.

Alexa

The API worked well, although it was a bit clunky to use, having to bookmark a page with the basic auth username and password built in.  This gave a warning on most browsers that you had to click through.  Ultimately a legitimate front end would solve this, but in the short term I decided to bring in yet another technology

I was aware that Alexa had an API to perform custom actions.  Setting it up took a few hours, mainly due to how cryptic Amazon is about everything they do.

First you need to create a lambda function.  Lambda functions are just short scripts you write in Javascript or Python and Amazon runs them when you hit some endpoint.  They're pretty straight forward.  I used Python 2.7, and set up a "role" (Amazon's permissions model) with whatever basic preset was available.  I then set the trigger to be "Alexa Skills Kit".  My code was just the color sample code, with all my code in the get_welcome_response method.  That method gets called when the Alexa runs the lambda and all I had it do was hit my API.

At this point you get an ARN which is what you need the Alexa to call to run your lambda.  The second half was much more confusing.  First, for some reason all the Alexa stuff is not in AWS, but rather the "Developer Console".  Once I found that I created a new Alexa skill.  There is a ton of configuration for the skills, but for the most part I either left it as defaults, or googled values to enter for things like "Intents".  The only real configuration I had to do was to enter my ARN as the endpoint, and enter what I wanted to say to turn it on as "Invocation Name".  Once I got to the testing step I enabled that and it worked.  I didn't have to fill out Publishing or Privacy details.

While the skill worked for me, I wanted to make it available to other users, while not actually publishing it.  After hunting around I discovered you can invite users to be developers in Settings > User Permissions.  They then have to accept, and go into the developer console and enable testing in the skill.  It will then show up as a custom skill in the Alexa app.

With this, the command "Alexa, laser" would turn it on/off, and it worked pretty well.  The only hiccups have been in Alexa failing to understand what is being said.

How's it work?

 This setup has been running for 3 months now, and has been amazingly robust.  There has been exactly one case of the API and client not working, and that was caused by the Pi losing its wifi connection for some reason and then failing to reconnect.  Unplugging the Pi fixed it.  I then set up another script to run on the Pi to check the last ping timestamp and restart the Pi if that is a few minutes old.


The code for the controller and client are on Github:

https://github.com/StephenWetzel/pi-controller

https://github.com/StephenWetzel/pi-client

Saturday, June 3, 2017

Network Protocols

https://www.destroyallsoftware.com/compendium/network-protocols
TCP has no special "I lost a packet!" message. Instead, ACKs are cleverly reused to indicate loss. Any out-of-order packet causes the receiver to re-ACK the last "good" packet – the last one in the correct order. In effect, the receiver is saying "I received packet 5, which I'm ACKing. I also received something after that, but I know it wasn't packet 6 because it didn't match the next sequence number in packet 5."

If two packets simply got switched in transit, this will result in a single extra ACK and everything will continue normally after the out-of-order packet is received. But if the packet was truly lost, unexpected packets will continue to arrive and the receiver will continue to send duplicate ACKs of the last good packet. This can result in hundreds of duplicate ACKs.

When the sender sees three duplicate ACKs in a row, it assumes that the following packet was lost and retransmits it. This is called TCP fast retransmit because it's faster than the older, timeout-based approach. It's interesting to note that the protocol itself doesn't have any explicit way to say "please retransmit this immediately!" Instead, multiple ACKs arising naturally from the protocol serve as the trigger.

Tuesday, May 16, 2017

MP3s as a litmus test for good journalism

The MP3 format was invented by a German group in the early 90s.  They patented it, and licensed it out to companies.  This is the reason many open source programs force you to download MP3 libraries separately.

The last patents for mp3 expire this year (2017).  Now anyone can use it without having to worry about licenses.  The group that created it announced they would stop licensing it (since they can't) and suggested people move to AAC (since they still own patents on that).

The result is news organizations running stories with headlines like "MP3 is Dead".  This presents and interesting look into which sources are reliable sources for tech news, and which use hyperbolic headlines for the sake of clicks. 

I went to Google News and searched for recent articles that mentioned 'MP3'.  Some of these were pretty obvious, but some were surprising.  To be fair, some are technically correct, in saying the creator declared it dead, vs saying it actually is dead, but merely parroting a press release is still going under the 'Bad' category.  The BBC was close, but I put it in good because it didn't feel clickbaity to me, feel free to disagree.

Finally, I won't pretend like this single example is some end all test for who you should and shouldn't trust, it's just and interesting source of some empirical data.

Bad

NPR: The MP3 Is Officially Dead, According To Its Creators
The Atlantic: The End of the MP3
Gizmodo: Developers of the MP3 Have Officially Killed It
The Register: MP3 'died' and nobody noticed
Quartz: Say goodbye to the iconic MP3
CNBC: The MP3 is dead, say creators after terminating licensing
The Telegraph: Creators of the MP3 declare it dead
Tech Radar: RIP MP3 - the sound file that changed the world is declared dead

Good

Washington Post: Your MP3s are going to be just fine
Mashable: The MP3 isn't dead yet, but it's now on its last digital legs
Vice: The MP3 Is Not Dead
CNET: MP3 isn't dead, it's just sleeping
BBC: It might be time to say goodbye to the MP3 - so let's look back at its life

Friday, May 12, 2017

Are Pop Lyrics Getting More Repetitive?

https://pudding.cool/2017/05/song-repetition/

This is some good data, but the presentation is very interesting as well.

Tuesday, April 25, 2017

Github has all the best cat stories

https://gist.github.com/lossyrob/a663186ccf37ea613a6b9ccb78de2cd1
In broad daylight, we could see why this was street cat utopia. What used to be a deli or some other food store collapsed in what looks like the 80s. A tree had grown through the inside where the roof had collapsed, a branch somehow punching through brick wall and completely enveloping a piece of old metal shelving. There was no way into this place past the first few steps. The roof was collapsed with a capital C. You could see through the busted rafters towards the middle of the (what was now) one big room of the first floor, and to the street cats that were lazily napping in the sun, protected by their fortress.

Tuesday, March 14, 2017

MediTech


I've been reading through these replies about "MediTech" trying to determine if it's some sort of elaborate inside joke I'm not picking up on.
There is a company called MediTech in Massachusetts that uses a derivative language of MUMPS called Magic. I know several programmers that have worked there. There are thousands of engineers writing in this language as we speak.
From what I can remember:

-Only global variables

-Variables must only be capital letters, maximum length 6. If you run out of variables, you must cleverly use them in a routine and set them back to what they are. This means you can't use a name like myVar - you use AAAFD, ZBVCXZ, etc.

-System functions are usually things like ., >, ', ], so code looks like .'AAAF]{\;:..

-Meditech writes all of their own languages, databases, operating systems, tools, etc. You can only write in a non-Meditech language if you get approval from a multi-tiered architectural design board, which barely ever happens

-The founder hated C with undying passion. No one is ever allowed to use C

-All programming hires go through a 6 to 12 month training process to learn the tools, languages, and systems. As they almost exclusively hire non-CS majors, such as math and physics majors, they don't typically have a programming background and don't realize how bizarre the MediTech stack is

Monday, March 6, 2017

A Good Overview of How Trump Operates


I try not to post a lot of political or topical stuff here, but this is a very good overview of Trump and how he operates.  It goes into a lot more background and detail than just the current Russia story.
Whenever he is under fire for something in a sustained way, he makes a shocking claim or provocative declaration about something else to change the subject. He is a master practitioner at the politics of distraction. These five examples might jog your memory:
  • After struggling during the first GOP primary debate to explain his disparaging comments about women, he attacked Megyn Kelly. “There was … blood coming out of her wherever,” he said, ensuring that the media focused on the new Trump-Kelly “feud.”
  • In November, the morning after agreeing to settle a fraud lawsuit against Trump University for $25 million, he demanded that the cast of “Hamilton” apologize to Mike Pence.
  • Perturbed when critics pointed out that he lost the popular vote, he claimed that 3 million to 5 million people voted illegally.

Saturday, February 11, 2017

Top mentioned books on stackoverflow.com

http://www.dev-books.com/

We analysed more than 40 000 000 questions and answers on stackoverflow.com to bring you the top of most mentioned books (5720 in total)

How we did it:
  • We got database dump of all user-contributed content on the Stack Exchange network (can be downloaded here)
  • Extracted questions and answers made on stackoverflow
  • Found all amazon.com links and counted it
  • Created tag-based search for your convenience
  • Brought it to you

Saturday, January 28, 2017

Overjustification effect

https://en.wikipedia.org/wiki/Overjustification_effect
The overjustification effect occurs when an expected external incentive such as money or prizes decreases a person's intrinsic motivation to perform a task. The overall effect of offering a reward for a previously unrewarded activity is a shift to extrinsic motivation and the undermining of pre-existing intrinsic motivation. Once rewards are no longer offered, interest in the activity is lost; prior intrinsic motivation does not return, and extrinsic rewards must be continuously offered as motivation to sustain the activity.

Sunday, January 15, 2017

The Line of Death

https://textslashplain.com/2017/01/14/the-line-of-death/
The Metro/Immersive/Modern mode of Internet Explorer in Windows 8 suffered from the same problem; because it was designed with a philosophy of “content over chrome”, there were no reliable trustworthy pixels. I begged for a persistent trustbadge to adorn the bottom-right of the screen (showing a security origin and a lock) but was overruled. One enterprising security tester in Windows made a visually-perfect spoofing site of Paypal, where even the user gestures that displayed the ephemeral browser UI were intercepted and fake indicators were shown. It was terrifying stuff, mitigated only by the hope that no one would use the new mode.