Pressure prediction analysis

Previously, I analysed how accurate various weather prediction services predicted the temperature a number of days in advance. Today I will look at their predictions for wind speed.

Here is the summary data

Here is a chart of the data for people who prefer visual:

Points to note

• Openweathermap seems to be the most accurate by far.
• Yr.no has similar accuracy openweathermap for one day in advance, but when predicting further into the future both darksky and yr.no are similar.
• The error from openweathermap seems to be much larger than other services. I checked and this error is genuine.

Like the last time, this analysis lumps all cities together for an average result. I might do some analysis later on performance per city.

Rainfall accuracy

In my previous analysis of forecast accuracy, I didn't look at the prediction for rain. Part of the reason for this is the fact that some of the services don't provide their rainfall reports and/or predictions in an easy to analyse format e.g. it might say 'light drizzle', 'cloudy' or heavy rain.

The one exception to this rule is the yr.no forecast service which provides rainfall predictions and reports in mm/hour. I thought it would be too strict to expect the forecasted mm/hour to match so I instead translated into rain or dry values.

This means that we can have 4 situations:

1. pred_rain => we predicted rain and we got it
2. unpred_rain => we didn't predict rain but we got it anyway
3. pred_dry => we predicted dry weather and we got it
4. unpred_dry => we didn't predict dry conditions but we got it anyway

I analysed 1 day and 7 day predictions for a selection of cities and I noticed a few key facts:

• Rain forecasts in Luxor are 100% accurate, but this is because it never rained at all in Luxor during our time to track. This is in stark contrast to the results from other cities I looked at.
• In Dublin, forecasts are about 82% accurate for the next day and 66% accurate for one week in advance.
• In Galway, forecasts are about 75% accurate for the next day and 60% accurate for one week in advance. This is only slightly better than random coin tossing which we would expect to  provide 50% accuracy

Here is the raw output if you want to look closer:

1 Days in advance for Dublin
Total 2316
pred_rain 240 10.36%
pred_dry 1662 71.76%
unpred_dry 216 9.33%
unpred_rain 198 8.55%

Correct 1902 82.12%
Incorrect 456 19.69%
 
7 Days in advance for Dublin
Total 2232
pred_rain 84 3.76%
pred_dry 1398 62.63%
unpred_dry 408 18.28%
unpred_rain 342 15.32%

Correct 1482 66.40%
Incorrect 492 22.04%
 
1 Days in advance for Galway
Total 2346
pred_rain 330 14.07%
pred_dry 1422 60.61%
unpred_dry 276 11.76%
unpred_rain 318 13.55%

Correct 1752 74.68%
Incorrect 606 25.83%
 
7 Days in advance for Galway
Total 2256
pred_rain 180 7.98%
pred_dry 1164 51.60%
unpred_dry 474 21.01%
unpred_rain 438 19.41%

Correct 1344 59.57%
Incorrect 654 28.99%
 
1 Days in advance for Luxor
Total 2346
pred_rain 0 0.00%
pred_dry 2346 100.00%
unpred_dry 0 0.00%
unpred_rain 0 0.00%

Correct 2346 100.00%
Incorrect 0 0.00%
 
7 Days in advance for Luxor
Total 2238
pred_rain 0 0.00%
pred_dry 2238 100.00%
unpred_dry 0 0.00%
unpred_rain 0 0.00%

Correct 2238 100.00%
Incorrect 0 0.00%
 
1 Days in advance for Coral Springs
Total 2364
pred_rain 708 29.95%
pred_dry 996 42.13%
unpred_dry 402 17.01%
unpred_rain 258 10.91%

Correct 1704 72.08%
Incorrect 1110 46.95%
 
7 Days in advance for Coral Springs
Total 2268
pred_rain 594 26.19%
pred_dry 846 37.30%
unpred_dry 498 21.96%
unpred_rain 330 14.55%

Correct 1440 63.49%
Incorrect 1092 48.15%
 
1 Days in advance for Perth
Total 2382
pred_rain 168 7.05%
pred_dry 1974 82.87%
unpred_dry 108 4.53%
unpred_rain 132 5.54%

Correct 2142 89.92%
Incorrect 276 11.59%
 
7 Days in advance for Perth
Total 2262
pred_rain 132 5.84%
pred_dry 1740 76.92%
unpred_dry 228 10.08%
unpred_rain 162 7.16%

Correct 1872 82.76%
Incorrect 360 15.92%

Accuracy of temperature predictions

Using the weather data I collected previously, I wanted to figure out which of the forecasts services was making the most accurate forecasts.

The way I calculated this was by comparing the temperature predicted in x days time with the actual temperature reported by the service. e.g. if the provider predicted on 1st/Aug that the temperature would be x degrees in 5 days time, then I compare this with their actual reported temperature on 6th/Aug, This absolute error is then averaged across all readings from  all cities

Here is the summary data

Here is a chart of the data for people who prefer visual:

Points to note

• I eliminated one reading from weather underground 6 day forecasts because it would have re-scaled the chart (probably indicates a programming error on my behalf).
• In general the forecast accuracy decreases as the number of days increases.
• Yr.no is best for next day, but bluemix and darksky are equally good further out.
• OpenWeatherMap and Weather underground are significantly worse.

This analysis lumps all cities together for an average result. I might do some analysis later on performance per city.

Weather forecast data

I have published several posts about weather. I also did articles with analysis of temperature and wind forecast, but there is a lot more analysis which could be done. Therefore I decided to publish the raw data in case anyone wants to analyse it. I will also do some more charting/plotting myself.

Click here to download the captured forecasts in CSV format. The file contains over 2 million records in total and the format is fairly self explanatory. It covers a period from June 2017 to December 2018, but some forecasts are missing due to a variety of factors. Most notably the Bluemix data only covers August 2017 to February 2018.

I will later post additional analysis files in this folder with some derived data from this original data in the CSV.

I recently found out about a professional analysis report comparing the accuracy of several forecast providers. I think you should value their results above mine.