Monday, August 29, 2011

How Irene Lived Up to the Hype

http://fivethirtyeight.blogs.nytimes.com/2011/08/29/how-irene-lived-up-to-the-hype/
So Irene right now ranks as the 10th-deadliest storm since 1980, with some possibility of that number going higher. And it ranks as the 8th most destructive storm economically, give or take. Meanwhile, it received about the 10th-most media coverage.

I found this storm rather annoying.  Not because it was overhyped, but because the hype was overhyped.  If you don't like hype don't watch the news.  Hype is their business, and business is good.

In the age of the internet there really isn't any reason not to get your information directly from the source for things like this.  Do you think Fox/CNN is operating their own fleet of weather buoys, or planes dropping sensor probes into the storm, or weather satellites, or supercomputers running simulations?  They are all getting their data from the NOAA, so you might as well too.

Sunday, August 28, 2011

Image Formats, And Which To Use

As someone who spends an enormous amount of time on the internet, one of my problems is when people use the wrong image format.  This leads to either too big a file size or a crappy looking image.  Which format to use is a pretty simple thing to figure out.  First some background.

JPEG is probably the the most common image format.  It is an old format.  While there are better choices, it is entrenched in many places.  JPEG will likely be around for quite a long time.  The key aspect of JPEG is that it is a lossy format.  This means that it loses information as a trade off for a smaller file size.  This is similar to how MP3s work.  JPEGs can have various quality settings which allow one to decide how much quality (and file size) the image loses.

GIF is also an old image format.  However, unlike JPEG it is lossless.  That means that no quality is lost when saving an image as GIF.  There are caveats to that though.  The biggest problem is that GIF only supports 256 colors.  This is fine for things like logos and icons, but any real world image will have a significant loss of color info.  Today GIF has largely been replaced by PNG.  However, one thing GIF has going for it is animation.  If you see an animated image on the web any time soon it's likely to be a GIF.

PNG was created largely to replace GIF, both due to patent issues and just general better quality.  Like GIF, PNG is lossless.  It overcomes GIF's main downfall by supporting either 24 bit or 8 bit color.  This means PNGs can either be smaller than GIF for similar color loss, or perfect color.  If JPEGs are MP3s then PNG is FLAC.

BMP is is a lossless and totally uncompressed image format used by MS Paint.  It produces huge files, which generally makes it the worst possible choice for anything.  Using our audio comparison BMP is a crappy version of WAV.

TIFF is another lossless image format.  This one is often used by scanners.  Like BMP, TIFF produced huge files and should be avoided unless you have a specific reason for using it.

SVG is different from the other formats described here in that it stores info about how to draw the image rather than what the image is.  The result is small file size and an image that can scale up infinitely (since the computer is drawing the image itself).  It is analogous to MIDI.  The downfall is that it uses more processing power, and isn't universally supported by web browsers.  In addition, it will only work for simple images that are collections of lines.  Wikipedia likes to use SVG a lot, much to my dismay.  Getting this image link crashed Firefox, twice.

Now that we've covered the common image formats it's time to answer the question: Which image format do I use and when?

For real world images (eg images taken with a camera), JPEG is the right choice.  PNG or TIFF could be used if you want no data loss at all, but you'll pay for it with a huge file size.

For images that were created on a computer (eg logos and icons), PNG is generally best.  If you have a heavily used image and want to minimize file size an 8-bit PNG is most likely going to be the smallest file size.  People often use JPEG for these types of images which leads to both greater file size and worse image quality.

To illustrate the differences I made a few example images:

Here is a screen shot of Wikipedia:
PNG Full 79 KB
GIF 58 KB
JPEG Low 53 KB
PNG 8-bit 37 KB


















The full PNG is a perfect image with no quality loss.  It's 79 KB.  The GIF and 8 bit PNG both lose some color info in exchange for a lower filesize.  The 8 big PNG is the smallest file I could get at 37 KB.  I've included a low quality JPEG (setting 15/100 in Gimp) to show the artifacting that JPEGs produce with high contrast changes such as you see in computer graphics.  Note in the 8 bit PNG that only the color gradient in the Windows title bar shows color loss.  All the color icons are fine.


Here is a computer created graphic:
PNG 8-bit 37 KB
JPEG High 183 KB
JPEG Low 46 KB











In this case the original image was a 47 KB GIF.  The 8 bit PNG will never lose color info over an 8 bit GIF and will almost always be smaller (about 25% smaller in this case).  The NOAA probably uses GIF for maximum compatibility (IE6 doesn't fully support PNG), and simply out of habit.  I recommend you open the PNG and the high quality JPEG in tabs and compare them.   The JPEG may look alright to begin with, but looking at the PNG shows the JPEG is quite blurry, losing data near any sharp edges.  This highlights why JPEG is a terrible choice for computer graphics.  Also keep in mind the JPEG is almost 5x as large.


Now here is an image taken with a camera:
JPEG High 124 KB
JPEG Low 28 KB
PNG 8-bit 230 KB









Really, the only choice here is a decent quality JPEG (85/100 pictured).  The low quality JPEG is much smaller but also horrible looking.  If file size is a concern you would probably want to go with something in the middle.  The 8 bit PNG was the smallest file of the various others I tried, and doesn't look very good.  A full quality PNG was 786 KB, with no noticeable improvement over the 124 KB JPEG.

Here are some tables with the files sizes (in KB) of the various formats I tried:

FormatKBNotes
wiki.png8bit.png37.4Color Loss
wiki.pngbw.png45.3B/W
wiki.jpglow.jpg51.7Artifacts
wiki.gif.gif58.3Color Loss
wiki.gifbw.gif65.3B/W
wiki.pngfull.png79.4Perfect
wiki.jpghigh.jpg182Slight Artifacts
wiki.bmp.bmp678Huge
FormatKBNotes
irene.png8bit.png36.7Perfect
irene.jpglow.jpg46.4Bad Artifacts
irene.gif.gif46.9Original
irene.pngfull.png66.124-bit
irene.jpghigh.jpg183Artifacts
irene.bmp.bmp627Huge
FormatKBNotes
catskills.jpglow.jpg27.6Low Quality
catskills.jpghigh.jpg124Good Quality
catskills.png8bit.png230Color Loss
catskills.gif.gif275Color Loss
catskills.gifbw.gif500B/W
catskills.pngfull.png786Perfect
catskills.bmp.bmp1671Huge

Something in the water

http://articles.boston.com/2011-08-21/bostonglobe/29912200_1_caleb-banta-green-drug-experts-new-drug
On March 4, 2008, Field and Banta-Green and other colleagues initiated a single-day study. Ninety-six Oregon municipalities agreed to take part in the research. They were large and small, rural and urban, representing 65 percent of the state’s population. The researchers took a portion of the daily flow from local water-treatment facilities and headed back to the lab, where they injected 2 millimeters of the sewage onto Field’s instrument and scanned it for meth, cocaine, and ecstasy. The same rigor that scientists apply to a new employee or an athlete taking a drug test, Field and Banta-Green were now applying to raw sewage. The results were very clear: The components of ecstasy appeared in less than half the treatment plants, cocaine’s components in 80 percent of them. The molecules of meth, though, were in all of them. The researchers published the study in 2009 in the journal Addiction to much academic fanfare.

Thursday, August 25, 2011

How to get $12 billion of gold to Venezuela

http://blogs.reuters.com/felix-salmon/2011/08/23/how-to-get-12-billion-of-gold-to-venezuela/
But here’s one last idea: why doesn’t Ch├ívez crowdsource the problem? He could simply open a gold window at the Banco Central de Venezuela, where anybody at all could deliver standard gold bars. In return, the central bank would transfer to that person an equal number of gold bars in the custody of the Bank of England, plus a modest bounty of say 2% — that’s over $15,000 per 400-ounce bar, at current rates.

Sunday, August 14, 2011

Why Haven't We Been Contacted By Aliens?

Consider the following: there are about 300 billion stars in our galaxy.  We have every reason to believe that these stars typically support a wide variety of types of planets.  In addition, there are hundreds of billions of galaxies, similar to our own.  Current estimates for stars in the visible universe are about 70 sextillion (7 × 1022).  Our solar system and Earth formed a relatively late 10 billion years after the universe began.  Other stars had formed and died by then.  Life on Earth formed a few hundred million years after the Earth formed.

Thus, it would seem there was both ample locations and ample time for life to have formed many many times other than on Earth.  Our universe and galaxy should be teeming with life.  With so much life, and with that life having a considerable head start on us, the unavoidable question is:
Why Haven't We Been Contacted By Aliens?

This question is known as the Fermi Paradox, and if you've never read about it I suggest you check out the Wiki article.  Here I will attempt to cover some of the major explanations for why there has been no contact with aliens.

Earths Are Rare
As I said above there are at least 7 × 1022 stars in the visible universe.  To date we've discovered over 500 planets outside our solar system.  However, those planets are mostly quite different from Earth, and unlikely to support life.  This is largely a result of the conditions that make planets easier to detect (very large mass, very close and short orbit).  We have only just begun to search for Earth like planets, but we have already found a few that could qualify.  We have no reason to think that Earth like planets are particularly rare.  Even if they were as rare as 1% of star systems, the absurd number of stars would mean there would still be an unimaginable number of Earth like planets.  I don't think that there is a lack of suitable locations.

Life is Rare
With only one example of a planet that supports life, we have very little idea of what kind of conditions are needed for life to begin.  There are a few things about Earth that are unusual, and if these things are necessary for life to begin it could mean that life just doesn't happen that often.  These go beyond a planet being Earth like (which usually means small, rocky, and having liquid water).  We have a large tilt giving us seasons, a large moon giving tides, and our star is located in a rather sparse area, protecting us from various cosmic catastrophes.  You can read some more of the interesting things Earth has going for it on Wiki.

In addition, while evolution does an excellent job of explaining how life can go from a basic replicating stage to the diverse complex array we are a part of today, we have no explanation for how life can spontaneously beginSome experiments have shown amino acids can form from sterile conditions similar to what would have been present on the early Earth.  However, while amino acids are an important component of life, they themselves are not life.

However, I don't think there is anything about the Earth that is that uncommon.  Nor is there anything that seems like it would be totally insurmountable.  Life on Earth began within a half a billion years of Earth forming.  A typical star system would last about ten billion years.  It seems like life began very shortly after it was possible on Earth.  Given our one example, I see no reason to assume life is particularly unlikely given reasonable conditions.

Intelligent Life is Rare
Many people have a flawed impression about evolution.  Specifically, that it leads to better and more advanced life.  Better is too broad a word to have much meaning, and while it certainly can lead to more advanced life, it isn't a foregone conclusion. 

Simply put, evolution is just a process where genes that cause themselves to be more likely to be passed on (by producing organisms better adapted to their environments, or more likely to successfully reproduce), tend to become more common.  These genes represent small changes, which over very long periods of time can lead to great diversity.  Note that neither 'better adapted to their environments' nor 'more likely to successfully reproduce', is equivalent to 'more advanced'.  More advanced may be a way to achieve those goals, but in and of itself is not a goal.

The evolution of human intelligence took 'only' 7 million years.  Life had existed for about a billion years, and complex land based vertebrates had existed for several hundred million years.  It seems like human level intelligence could have evolved earlier if it was a significant evolutionary advantage.

Another evolutionary misconception is that improvements in speed, size, strength, intelligence, etc are advantages.  While these things could be an advantage, they are trade-offs.  They come with an increase in calorie demands.  If the improvement doesn't allow for a corresponding increase in calorie intake, then it is a disadvantage.  The human brain is no exception to this; it consumes about 20% of the calories we take in.

It is not hard to imagine how the increased intelligence of humans may not have been enough to offset the increased caloric demands that went along with it.  Would a human, without the accumulation of knowledge passed down, be any better at surviving than another great ape?  The fact that we are here would seem to be evidence that the increased intelligence would be an actual advantage.  However, there could have been a unique set of selective pressures that caused intelligence to be beneficial.  If this is the case, it could be that diverse complex life is common, but that intelligence is rare or unique. 

Intelligent Life Tends to Destroy Itself
The ability of the human race to destroy itself is a unique and recent development.  More than a hundred years ago it wouldn't have been possible for any group of humans to wipe out the entire human race.  With nuclear weapons, this became possible.  There are other methods that would have the same result as well, some that could be accidents.  It could be argued that as intelligent life develops more and more technology the number of ways it could destroy itself only increases.  Eventually it becomes certain that one of these things will happen.

That being said, I've never been a fan of this explanation.  It would seem we are unlikely to wipe ourselves out anytime soon, and we have already reached a point where we are detectable from space.  In a hundred more years it seems reasonable we could be taking the first steps into other solar systems.  Once we reach that point, I would think we'd be a lot harder to wipe out.  Again, with the number of stars it would only take a very small percentage of civilizations that didn't wipe themselves out to lead to a huge number of civilizations that should be present.

Faster Than Light Travel is Impossible
While simply accelerating to the speed of light is impossible there are a number of 'loop-holes' that would allow for travel between two points in less time than light would take.  If it turns out that even these are impossible and that it is truly impossible to travel astronomical distances faster than light then this would pose a significant problem for galactic colonization.  There could be a few dozen current civilizations in our galaxy that we'd never contact due to the distances.

That being said, it would take at least 100,000 years to cross our galaxy.  If there were thousands or more advanced civilizations it seems probable that some would be traveling around at close to the speed of light, and that some would have come across the Earth by now.

These are just the main reasons I think are most likely to be relevant.  Wikipedia goes into much more depth.  In addition there is something called the Drake equation, which aims to estimate the current number of advanced civilizations given a set of variables.  If I had to make a guess as to the reasons, I'd have to say a combination of rarity of intelligent life and faster than light travel being impossible.

Sunday, August 7, 2011

Firefox Extensions

I started using Firefox a long time ago, back when it was Phoenix.  It was a great day when I discovered it.  The superiority to IE was unquestionable. I'm not sure if I could have spent every waking moment of my life on the internet without it. The two main features that drew me in right away were tabs, and extensions. Tabs are now the norm, and while other browsers allow extensions, Firefox has the advantage of the huge catalog of extensions available already.

Now it seems Firefox's momentum has slowed. While IE continues to be abandoned, Chrome has risen to capture those users. One of the benefits of Firefox was that it was fast and lean. It was just a basic web browser. If you wanted more, that's what extensions were for. Unfortunately, Firefox has fallen victim to the same problem that every successful program has. Feature creep is inevitable in a popular program. The simple fact is, it's more fun to add new features rather than sit around fixing bugs.

Still, I have grown dependent on some extensions, and will likely not switch away from Firefox for a while. I decided to write a post about what extensions I use, so that people can be more like me. I have a page on my site about the programs in general I use, so feel free to install all them too.

Without further ado, here are my currently installed Firefox extensions:
Adblock Plus - This pretty much goes without saying.

Download Statusbar - I've used this for years. Makes downloads just show up in a status bar, instead of a box.

FlashGot - I installed this years ago first as a way to build galleries from predictable image names (ie sequentially numbered images). Now I use it to do two things. First, download all links with a certain extension. Second, to download embedded flash videos (eg Youtube).

Greasemonkey - If you don't know what Greasemonkey is, it is pretty interesting. It is a javascript engine that lets you run scripts on certain pages. In effect it allows for small simple extensions that only run on certain pages, which mainly change the look of sites. I've written a handful of custom ones. Some of the others I use are: Show Just Image - which auto forwards to just the full sized image on the endless array of image hosting sites. Super iGoogle, which removes the worthless stuff (sidebar) from the iGoogle page. And xkcd titles, which displays the "secret" alt text under the image rather than requiring you to hover over it.

Multi Links - Multi Links is an interesting recent addition. It allows you to right click and drag a box which will open any links you cover in new tabs.

NoScript - If you're unaware of NoScript, it blocks javascript from running on certain sites. You can either have it run by default on everything and blacklist sites, or block it by default and whitelist sites. It's great for security and blocking ads and stuff that would autoplay sounds. It's also funny how many sites have some sort of security that is entirely javascript and bypassed with NoScript.

RefControl - RefControl lets you hardcode your referral string, which lets websites know what site you are coming from. I have no idea why I have it installed, no doubt trying to bypass some restriction somewhere.

StumbleUpon - StumbleUpon is probably the greatest invention since the internet. You tell it what interests you have, then click the stumble button and it displays sites that fit those interests. You can thumb up or down sites and that determines how often they are shown to other people, as well as further determining what types of sites you like. Essentially it's a never ending stream of mildly amusing or interesting content.

Tab Mix Plus - Tab Mix Plus allows a ton of customization for how tabs behave and look. Some things which I think it has allowed (some of these may be default behavior): Allow tabs to be very small, and take up multiple rows, great for when you have 100 tabs open. Make new tabs open at the end of the list, instead of near the current tab or whatever the hell the crazy default is. Reload or close all tabs either to the right or left of current tab.

Those are all the extensions I have currently installed. Here are ones that I've had installed previously, but have determined I don't need.
Firebug - A great developer tool. Allows you to examine the CSS and JS on a page and debug it step by step.

Forecastfox - Displays weather in status bar. Don't need it since I've discovered WeatherSpark, and check it 20 times a day.

Nightly Tester Tools - Pretty good set of advanced tools. Really the only things I ever used was the ability to copy extensions to clipboard, and display the build in the title, which does nothing, but makes me feel cool.