How To Predict The Future

A while ago, I noticed a trend when people made predictions for the future.  Short term predictions tended to be too short, while long term predictions tended to be too long.  For example, some new technology that you think will be around in 10 years will probably be closer to 20 years.  On the other hand, if you think a technology will take 500 years it'll probably be something like 250.

I think there's relatively simple explanation for this.  People tend to focus on big changes and overlook small ones.  If we take a technology like solar power, there are plenty of predictions about it being cheap enough for large scale use in 10 years.  However, these predictions likely overlook the various small problems with implementation.  Perhaps we will have solved the largest problem keeping costs high in 10 years, but there will be a number of other problems that we haven't solved, many of which we aren't even aware of yet.

It is even clearer in long term predictions.  Look at a sci-fi story from a few decades ago, and you will see lots of technology that is inferior to today's tech.  A well know example of this is the original Star Trek series from the 60s.  They use big clunky controls, and the portable devices are already inferior to a tablet computer.  Keep in mind it is set over 200 years in the future, so there is little doubt that we will be massively more advanced than them by the same time.  The exceptions to this would be single big inventions that may well be impossible (eg faster than light travel, matter transport).

When someone is writing a story based in the future they need to come up with each invention.  It's easy to come up with major inventions (particularly if they are needed for the plot), but most inventions aren't major.  Most inventions are small and evolutionary.  In the last 50 years there have been a few major inventions (and there is no hard line, so it's futile to try to decide exactly what has been major vs minor), but there have been millions and million of small innovations.  Most of the change in our daily lives is from these small innovations vs the major revolutionary inventions.  On top of this, most of these innovations have been incremental, ie, they relied upon the previous innovations to exist first.  Without coming up with the long chain of innovations, it would be impossible to predict the one at the end.

Getting back to my main point, short term predictions need to be lengthened, while long term predictions need to be shortened.  When I come up with a prediction I just double it if it's < 50 years, or half it if it's > 200 years.  This is ok, but I figured there had to be a better way to make the adjustments.  I wanted to come up with some formula that would tend to move predictions towards a certain time frame (eg 100 years).  I had some requirements.  First, it should tend to about double short term predictions, and half long term ones.  Second, as the year gets closer to 100 the adjustment should become relatively less, with the adjusted year never being moved past 100 years.  Last, I wanted one formula for < 100 and > 100 years, not a piecewise function. 

I came up with a formula to do this a few years ago, but didn't save it.  My recent reading of Manna made me think about this again.  So, I started a spreadsheet and set out to come up with a new formula that fit my requirements.  After some tinkering I came up with this:$$$\cdot$$$ $$$\frac{}{}$$$
$$N=I^{\sqrt{\frac{2}{\log{I}}}}$$
Where: N is the new outputted year, I is the inputted year, and log is the base 10 log.  And if it's not clear, the sqrt is an exponent of I.

This worked out pretty well.  It's a bit too high for input years < 5 years, and an input of 1 year causes a divide by 0.  10 years gives 26, 25 gives 47, 50 gives 70, 150 gives 122.  Looking at the adjusted values though, I think the turning point is too far out.  More thought convinced me that something like 70 years was probably better.  Changing this was rather easy.  I used 2 because log₁₀(100) = 2.  So it followed I could just replace 2 with log₁₀(t).  This had the additional benefit of making the base of the log not matter (as long as you use the same base on the top and bottom)
$$N=I^{\sqrt{\frac{\log{t}}{\log{I}}}}$$
Where: N is the new outputted year, I is the inputted year, t is the year where the turn from adding years to subtracting years happens, and log is any base log.

Looking at the formula I noticed something interesting.  If you're unaware, calculators commonly perform base 10 or base e logs.  If you want to perform a different base log you need to use the following formula:
$$\log_b{x} = \frac{\log{x}}{\log{b}}$$
Where: log_b(x) is the base b log of x, and log is any arbitrary base log.

In other words, if you want to find the base 7 log of some number, but can only compute the base 10 log then you have to find the base 10 log of whatever number you want to find the base 7 log of, and then divide by the base 10 log of 7.

With this knowledge, it should be clear that the above formula is really:

$$N=I^{\sqrt{\log_I{t}}}$$
Where: N is the new outputted year, I is the inputted year, t is the year where the turn from adding years to subtracting years happens, and log_I is base I log.

I attempted to get the variables out of the exponent using more logs.  I failed, but managed to get it in a different, and I feel interesting, form:
$$N = I^{\sqrt{\log_I{t}}} = b^{\sqrt{\log_b{t} \cdot \log_b{I}}}$$
Where: N is the new outputted year, I is the inputted year, t is the year where the turn from adding years to subtracting years happens, and log_b is the base b log.

If we use a base of 10 for the logs, and a turning year of 100 this becomes:
$$N = 10^{\sqrt{2 \cdot \log{I}}} = 10^{\sqrt{\log{(I^2)}}}$$
Where: N is the new outputted year, and I is the inputted year.

This formula is about as simple as I can make it. More importantly than that though, it reveals some interesting things. First, note how there are two functions, log and sqrt, and then their inverses, 10^x and x^2. They are nestled inside each other in an alternating fashion. Second, note that the conversion is now simply a power of 10. This means you can just focus on the exponent part to get a feel for how it will behave.

Anyway, I made two graphs showing how the function translates years. First is specifically the input years < 100. The second ranges 0-10,000.

 

In the second graph it's clear that it's similar to a simple log graph.  I was worried that something as simple as log(I) could produce an adequate curve.  A simple log(I) hugged the x axis too much, but multiplying by 100 steepened the curve.  Then, subtracting 100 shifted it down so that the graph went through the point (100, 100).  This looked ok for the low values, however, it flattened out too quickly for the higher values.  I played around with the values some more, but couldn't find any way to get the curve right.

So, I'm pretty happy with my formula.  Maybe I'll think about it some more and try to figure out if there is a simpler way to get that curve.  After all, I always need ways to avoid studying the math that I'm actually going to be tested on.

If you want some more interesting info on the formula, Wolfram|Alpha never ceases to amaze:
http://www.wolframalpha.com/input/?i=10^sqrt%28lg%28x^2%29%29