Tornado History Project - Over 50,000 tornado maps. Statistics from 1950-2007.

One of my favorite recently added features of the site (see Major Upgrade to Tornado History Project 7/19/08) is the ability to easily correlate each tornado in the Tornado History Project database to the original line (or lines) in the Storm Prediction Center (SPC) archive. Why is this important? It is not uncommon for users to email me with a comment such as:

The county listed for tornado X is wrong. It should be Y - I know because it hit my house. Why don’t you do a little research next time before posting this junk to the Internet!

While I do want to know when there is a suspected error in the SPC archive, I don’t enjoy being personally blamed for it. :-)  So, this will make it much easier to show users that the Tornado History Project is accurately reporting the data as given in the original SPC archive. Or, conversely, it will be easy to tell if I have made an error in translating the data.  In short, it adds transparency to the site and takes the mystery out of where the data comes from.

Let’s take a look at where you can find this info. On any individual tornado map page, all stats for that tornado are shown in an easy to read format immediately under the map as shown in the picture below. The last line of the section shows the actual text from the SPC archive used to generate those stats.

Of course, being able to actually see the original line from the SPC archive isn’t quite enough. It is also important to know how to read and interpret the information. So, a link is also given to the SPC instructions that explains what each piece of information actually means.

I should point out that there is one situation in which the approximate path shown for a specific tornado may not actually be based on the data in the SPC archive. Please see a previous blog entry (Enhanced Paths Explained 7/19/08) for more information.

And finally, there is one more note on this…Users must be logged in to see the corresponding line (or lines) from the SPC archive, so if you don’t see anything it, just login or register for a free account and you’ll be all set.

One of the most exciting new features of the site (see Major Upgrade to Tornado History Project - 7/19/2008) is the inclusion of enhanced path data derived from the National Climatic Data Center (NCDC) historical archive (thanks to Stuart Hinson for providing access to the archive!) While there are many differences between the Storm Prediction Center (SPC) historical archive (the primary source for the site) and the NCDC archive, the most significant difference is the method used to give coordinates.

In a nutshell, for tornadoes in which path data is available, the NCDC archive gives latitude and longitude coordinates when a tornado enters another county, while the SPC archive only gives coordinates when a tornado enters another state. This means that multi-county tornadoes in the NCDC database may have more data points available to draw a more detailed approximate path on the map. Thus, instead of the straight lines generated for SPC derived paths, NCDC derived paths generally have more turns and zig-zags.

How big of a difference does the inclusion of NCDC data make? Well, that depends on the number of NCDC derived data points given for any specific tornado, but let’s look at an example. The screenshot below shows a few F5 tornado paths using SPC data. As you can see, the approximate paths are straight lines, except for where a tornado crosses a state line.

The screenshot below shows the same area using NCDC derived data, when available. As you can see, some of the paths show a greater amount of detail.  Compare the apparent path near Jackson, Mississippi, for example.  With the enhanced path data below, the path passes the city from the South rather than the North.  And the more you zoom in, the more the difference will become apparent.

Displaying enhanced path data was not an easy task and took me almost a year to complete. The main issue was that there was no easy way to match up the tornado records between the two historical archives. I had to do quite a bit of manual tweaking to get it to work. Nevertheless, I think it was well worth the effort. I haven’t done an exact count, but I estimate that there are roughly 2,000-3000 tornadoes with enhanced path data, out of the 50,000 tornadoes in the database (the vast majority of tornado paths are short and only affect a single county.) Please note that you can only see enhanced path data when you are logged in.

I believe this to be the first time anything like this has been done. If someone knows different, please let me know. In any case, I hope you will sign up and do your own tornado search.

It has been a long time coming and is long overdue, but I have finally released a significant update to the Tornado History Project. For those who have been here before, the first thing you might notice is a change to the general look of the site. In short, it just looks better. But that is minor compared to the new features. I’ll post more in-depth details on some of the new features later, but the major ones include:

• Historical data extended through 2007
• A user account system
• Enhanced tornado path data for select tornadoes (see Enhanced Paths Explained 7/19/2008)
• Ability to correlate any tornado with the related line in the original SPC archive
• Ability to export custom data searches
• New buttons on the map, including experimental integration with Google Earth
• Improved functionality in the tornado forums (there are over 50,000 forums available.) Improvements include:
  • Immediate posting of user comments (no need to wait for them to be approved)
  • Ability to watch any tornado forum for comments
  • A user ranking system (F0 through F5)
• Other minor improvements

While these improvements are important, their real value is that they are the necessary foundation for future development. Keep checking back as I hope to improve the site even further. And remember, the purpose of this site is to recreate the history of as many tornadoes as possible through eyewitness accounts, personal memories, photos, and videos. I can’t do that alone - I need your help. So, tell your friends about this site, post your own memories or photos and videos, and, if you can, make a small donation.

It recently came to my attention that the time given for tornadoes occurring in 2004 was incorrect. The root of the problem is that the time as given in the source data (SPC historical archive) are in a different format for 2004 than in all other years. Thus, the standard translation used for all other years does not work for 2004. To make a long story short, the Tornado History Project database has been updated with corrected 2004 tornado times.

The latest update to the Tornado History Project has made searching for tornadoes easier than ever. Previously, there were separate pages for searching the database and outputting the results on a map or in a table. So, if you were viewing the search results in table form, but wanted to view the same search on a map, you had to 1) go to the map page - which erased your previous search parameters 2) re-enter your search parameters 3) submit your search. Now, a user can simply use the combined search page and toggle between the map and table outputs.

Also, in previous posts I have mentioned that improved path data was forthcoming for selected tornadoes. This is still in the works, but has not yet been completed. I will post here when it is ready to go.

The 2006 data has finally been added! I know I initally said I’d be done with the 2006 update by the end of September, but it proved to be more difficult than I had anticipated. Well, actually, getting the data added wasn’t the difficult part - it was the near complete internal database reorganization I decided to do. To make a long story short, there are several inconsistencies within the SPC archive. I’m not talking about errors in the data such as incorrect Fujita classificaions, lat/lon coordinates, etc. (there are still many of those…), but rather, inconsistencies with how segments of the data relate to one another. I have identified most, if not all, of the inconsistencies and have made modifications where needed. The modifications were generally limited to the internal SPC numbering scheme (the “SPC Tor. #” and “State Tor. #” columns on the Tornado Database page) and do not affect the official stats such as locations, fatalites, injuries, etc. Any modified records will be displayed in a red font when viewing search results on the Tornado Database page.

Also, significantly, I have completely changed the internal tornado indexing scheme. This shouldn’t affect anything, but it is an important change in that the new index will serve to permanently identify tornadoes in the Tornado History Project database going forward. The index is derived by combining the date of the tornado, the FIPS code of the state it occurred in (or the first state it occurred in if it is a multi-state tornado), and the state tornado number of that tornado. So, a tornado that occurred on 1/1/1950 in Alabama (FIPS code=1) and has been given the state tornado number label of 1 by the SPC will become tornado index #19500101.1.1
In addition to internal database changes, I have made the site somewhat easier to use (I think…) by providing clearer instructions and more appropriate icons representing links to maps, user comments, photos, and videos. You can see this by taking a look at the Tornado Database page. Tornado paths have also been changed to match the color of the tornado icon. So, an F3 tornado is now represented by the old orange icon and the new orange path line (instead of a standard blue path line…)

A few other changes were made too, but probably not significant enough to discuss here. Keep checking back over the next few weeks/months as I have a few other updates planned. The biggest update will be adding more detailed path data for many tornadoes (see previous blog post for more information.

The SPC (Storm Prediction Center) has recently added 2006 data to their historical tornado archive . A quick glance through the documentation reveals 2 new data elements have been added:

• Pl - A categorization of path length calculated as Pl=2*log(Length) + 1
• Pw - A categorization of path width caculated as Pw=2*log(Width) + 5

Additionally, 2 data elements present in past releases have been removed:

• DPI (Destruction Potential Index)
• Code (the meaning of this element was unknown)

Over the next few weeks I’ll be comparing the 2006 file with the 2005 file to see if any changes have been made to past (pre 2006) data. Once everything is squared away, I’ll add 2006 data to the TornadoHistoryProject website. Hopefully, this will be done by the end of September (or before), so keep checking back.

I have also obtained a separate historical archive from the NOAA (National Oceanic and Atmospheric Administration) that includes more detailed path information for multi-county and multi-state tornadoes (I am not yet sure of the derivation of this archive; I’ll post more when I have more information…) Essentially, lat/lon coordinates are given when a tornado passes from one county or state to another. This will result in a more accurate representation of tornado paths (particularly for long paths.) Having said that, there are a number of discrepancies between the two historical archives that need to be resolved. I may end up adding the expanded path data in separate year by year releases as each year’s data is cross-checked between the two databases. However I do it, this will definitely be a labor intensive project so it may take a while to fully complete it. I’ll post more details as I go along.

• Late 2005 - Tornado History Project concept is born when Josh Lietz simultaneously stumbles upon:
  1. Text archive of raw historical tornado data published by the SPC (Storm Prediction Center.)
  2. New Google Maps application
• Early November 2006 - Tornado History Project goes online, making historical tornado data freely available to the public in an easy to read, user-friendly, and searchable format.
• November 2006 - Tornado Map completed
• February 2007 - Added ability for users to add links, personal histories, etc., to any tornado
• March 2007 - Tornado map icons updated to visually distinguish tornadoes by Fujita rating

Welcome to the new Tornado History Project News/Blog. This area will be used to post updates to the site and to serve as a record of changes. Check back soon as several updates are under development. More details to follow…