So I was going through my flash price spreadsheet, and I noticed the cheapest price from when I stopped updating it in Nov-07 was $6/GB. Today it's $1.50/GB. So I thought about how I wished I had kept it up, as I'd like to see if there was any pattern. Also I again tried to find any kind of record of Newegg's prices, and couldn't. I decided now that I have the ability to scrap web pages I'd use that to automate it. So I made a batch script to download the search results for 1-32GB, and then .7z them. That was going to be it, I figured I'd set that up to run once a day and then have a nice archive in a year. But, then I decided to just make the perl script to parse the plain text and extract the price, model, and type and then append that data to the end of a csv file. Now I had a spreadsheet that updated daily and had the prices for the different sizes. Then I really wanted some way to upload it to the web automatically. I figured I could get something that would sync it with a google doc spreadsheet, but no such luck. Then I found some command line tools to upload stuff, but it was all python, and I don't have python installed, still looked promsising for the future.
Well today I figured I could just output a html page and then ftp that to my site via batch. It was going to be a simple html page, but I forgot perl is used to make html all the time, so it had good support for html formatting. So now it finds the cheapest price and outputs a page saying what it is, with a link to a newegg search for the model number (which should go directly to the product page, but I made make it grab the newegg item number and make the direct url if that doens't work in the future). Then it just ftp's it to my site. It all runs from my laptop, and considering the daisy chains of programs, batch files and scripts that make it runs on it works quite well and fast.
I'd prefer to run it on my mom's computer, as that is on more often (although the laptop is virtually always on). But it requires lynx, 7zip installed to command line, and perl. I'll probably install them on there later. It updates at about 825am every day.
http://daleswanson.org/things/flash.htm
No comments:
Post a Comment