TheBluesBrother
Hi Mike, I sure hope this works - I only just learned (I hope) to use Dropbox so you can access this stuff,
which is another first because prior to that I had to learn how to turn my Python code into an exe style program that you can run without installing Python, which was the result of learning how to 'scrape' code from the Timeform site.... if all goes according to plan......
This link should, if I understood this correctly, allow you to download a file called scraper.zip that I made up earlier. It isn't the complete program, I want to make sure that this is all going to work as planned before I complete the data grabbing side of things, so I put together a sort of little demonstrator program that collects the race times for yesterday from Timeform, then lists them in a csv file you can load into Excel. My plan is to do the processing so all you need to do is look at the results, but there'll be nothing to stop you playing with the data in Excel if you want to do other things with it. So if you'd download and unzip the file, then see the instructions that follow below:
Dropbox - scraper.zip
When unzipped (into any convenient folder you like) you'll see a small collection of files, a dozen or so, plus a folder called 'library' - there's also a file called 'scraper' which is listed as an application, you double click it to run the program. The other files are needed to run the code, as are a number that should already exist in your Windows directory on your PC - errors could come up if they aren't there, but they should be.
When you double click 'scraper' a command window (a black 'DOS' style job) will appear for the duration of the run, this window will click back out of existence when it finishes its job.
In that window you'll see messages (mainly to reassure you it is working) saying it's found so many meetings, and that it is checking the times for each race in turn. When it finishes running it will say "Where did it go?" - This is something I added to stop the command box winking out of existence on exit, press return and the box will disappear. (ie It's just an 'I'll press a key when I'm ready' sort of thing...without this I found it a bit disconcerting that everything just winked out of existence as it was getting interesting...)
That's the program finished - if you look in the folder your scraper program is in you'll see a file called Timeform_racetimes.csv (there's one from today in the zip file as an example) - open it in Excel or a similar spreadsheet program and you'll have three columns - racecourse, race start time, racetime.
If this all works okay for you then I can get on to finish the rest off, which should be a bit easier than getting this first bit done as I've got most of the 'skills' I need for the job now thanks to the last couple of days efforts. If all is okay I'll add the Racing Post times to the spreadsheet, and add a small menu to let you pick which date to get the times for. Would you like the race times converted to seconds as well? Currently they're in as '4m 16.5s' and similar, it's easy enough to program in a conversion routine to add a column saying 256.5
As you want to compare RP to TF I presume you'd like a column listing the time difference between the two racetimes listed for each race.
Is there anything else you'd like in there, basically if it's on the web page the times are on I can probably collect it. The program actually has to load a web page for each race individually with Timeform, so the speed it all runs at is down to your internet connection really - for example the test runs I did today loaded one page to collect the page addresses for each race, then looped through 38 individual pages to collect the time off each, which took maybe 2 minutes over sluggish ADSL.
Let me know if it works, and I'll get the rest done.
Dave
