I've been monitoring the BBC's 5 day forecast for a couple of months
now and since we've got to the end of May, I thought people might be
interested in just how accurate it's been.
What I've done is look at the forecasts and compared them to each other.
I haven't correlated whether the weather they forecast actually happened,
just how often the forecasts changed (since a 5 day forecast that
changes the very next day can't be relied on, so isn't any use). I've
also only used the forecast for my particular location.

So without further ado, here's the data, the explanation follows:

forecast forecast on: times num of days
-- date -- on day D-1 D-2 D-3 D-4 correct changes ahead
Sat 1 May HRS* HRS HRS LR LRS 2 2 2
Sun 2 May LR LRS HR HR SI 0 3 0 !
Mon 3 May SI SI SI S S 2 1 2
Tue 4 May SI* SI SI S S 2 1 2
Wed 5 May SI* SI WC SI SI 3 2 1
Thu 6 May WC SI WC SI S 1 4 0
Fri 7 May SI LRS WC SI LRS 1 4 0
Sat 8 May LR LR LR LR LR 4 0 4
Sun 9 May WC SI SI GC LR 0 3 0 !
Mon 10 May SI S SI SI WC 2 3 0
Tue 11 May SI* SI SI LRS D 2 2 2
Wed 12 May LRS LRS S SI LRS 2 3 1
Thu 13 May LRS* SI SI S S 0 2 0 !
Fri 14 May WC SI SI HRS GC 0 3 0 !
Sat 15 May LRS SI SI SI LRS 1 2 0
Sun 16 May LR LR LRS SI SI 1 2 1
Mon 17 May LRS LRS S SI SI 1 2 1
Tue 18 May SI S LRS SI SI 2 3 0
Wed 19 May SI* SI LR LR LRS 1 2 1
Thu 20 May SI* GC SI SI SI 3 2 0
Fri 21 May SI* SI S SI SI 3 2 1
Sat 22 May S SI S S SI 2 3 0
Sun 23 May S* S S S S 4 0 4
Mon 24 May S* SI S S SI 2 3 0
Tue 25 May WC SI SI SI GC 0 2 0 !
Wed 26 May SI SI WC SI SI 3 2 1
Thu 27 May SI* SI SI SI LRS 3 1 3
Fri 28 May LRS SI SI SI LRS 1 2 0
Sat 29 May HR LR LR HR S 1 3 0
Sun 30 May SI SI SI SI SI 4 0 4

For last 30 days: avg 2.13 different forecasts per day
On average forecasted 1.00 days ahead
Correct 4 days ahead: 3 times
Correct 3 days ahead: 4 times
Correct 2 days ahead: 8 times
Correct 1 day ahead: 15 times (50.0%)
Number of times the weather was same as yesterday(*): 11 times (36.7%)
Number of days every forecast was wrong(!): 5 times (16.7%)

Key:
D: Drizzle 0 occurrences, predicted 1 times over-forecast
GC: Grey Cloud 0 occurrences, predicted 4 times over-forecast
HR: Heavy Rain 1 occurrences, predicted 3 times about right
HRS: Heavy Rain Shower 1 occurrences, predicted 3 times about right
LR: Light Rain 3 occurrences, predicted 11 times about right
LRS: Light Rain Shower 5 occurrences, predicted 14 times under-forecast
S: Sunny 3 occurrences, predicted 21 times over-forecast
SI: Sunny Intervals 13 occurrences, predicted 58 times about right
WC: White Cloud 4 occurrences, predicted 5 times under-forecast

-----------------
If it doesn't appear in straight columns, you should view it with a fixed width font.

Now to explain. There are 5 forecasts for each date. D-1 means the forecast made
on the preceding day. So for example, on May 18 the forecast made on that day
for the weather was Sunny Intervals. The previous day (D-1) the forecast for
May 18 had been Sunny and the day before that (i.e. on May 16) the forecast for
two days hence had been Light Rain Showers.
I've assumed that a 5 day forecast is correct if, on day D-4, it predicts the
weather on a particular day AND that the forecast doesn't change in the intermediate
days. As you can see, this hardly ever happens :-( and a random guess wouldn't
be any worse. (In fact if the forecasts said "Sunny Intervals" every day
they'd be better than the ones produced, right up to the day before).
It's also noteworthy that the accuracy seems to decrease by about 50% each day:
15 - 8 - 4 - 3, given the limits of integer counting.

It seems to me that the basic problem with the forecasts is that they try for too
much detail, too far ahead. So while the "on the day" forecast is probably right-ish
to try to say what the weather will be in 4 days time, to the same level of detail
doesn't work. The understanding and even chaotic elements don't allow it. If you
use less discriminating terms, such as just trying to say if it will be wet or dry
the accuracy goes up quite a lot, Thus:

(excerpted)
forecast forecast on: times num of days
-- date -- on day D-1 D-2 D-3 D-4 correct changes ahead
...
Mon 10 May Dry* Dry Dry Dry Dry 4 0 4
Tue 11 May Dry* Dry Dry Wet Wet 2 1 2
Wed 12 May Wet Wet Dry Dry Wet 2 2 1
...

For last 30 days: avg 0.73 different forecasts per day
On average forecasted 2.73 days ahead
^^^^
Correct 4 days ahead: 15 times
Correct 3 days ahead: 19 times
Correct 2 days ahead: 22 times
Correct 1 day ahead: 26 times (86.7%)
Number of times the weather was same as yesterday(*): 21 times (70.0%)
Number of days every forecast was wrong(!): 1 times (3.3%)

Even so. it seems that 2 - 3 days is about all it's possible to forecast
in England.

"pete" wrote in message
...
I've been monitoring the BBC's 5 day forecast for a couple of months
now and since we've got to the end of May, I thought people might be
interested in just how accurate it's been.


Where and when is 'the BBC's 5 day forecast' ?

Tom

On Mon, 31 May 2010 15:21:01 +0100, Tom Allen wrote:

"pete" wrote in message
...
I've been monitoring the BBC's 5 day forecast for a couple of months
now and since we've got to the end of May, I thought people might be
interested in just how accurate it's been.

Where and when is 'the BBC's 5 day forecast' ?

Tom

This is the link for London.
You can personalise it for your postcode.
http://news.bbc.co.uk/weather/forecast/8?area=London
--
MCC

On May 31, 12:46*pm, pete wrote:
I've been monitoring the BBC's 5 day forecast for a couple of months
now and since we've got to the end of May, I thought people might be
interested in just how accurate it's been.
What I've done is look at the forecasts and compared them to each other.
I haven't correlated whether the weather they forecast actually happened,
just how often the forecasts changed (since a *5 day forecast that
changes the very next day can't be relied on, so isn't any use). *I've
also only used the forecast for my particular location.

So without further ado, here's the data, the explanation follows:

* * * * * forecast *forecast on: * *times * num of *days
-- date -- on day * D-1 D-2 D-3 D-4 correct changes ahead
Sat *1 May *HRS* * HRS HRS *LR LRS * *2 * * * 2 * * *2
Sun *2 May * LR * *LRS *HR *HR *SI * *0 * * * 3 * * *0 !
Mon *3 May * SI * * SI *SI * S * S * *2 * * * 1 * * *2
Tue *4 May * SI* * *SI *SI * S * S * *2 * * * 1 * * *2
Wed *5 May * SI* * *SI *WC *SI *SI * *3 * * * 2 * * *1
Thu *6 May * WC * * SI *WC *SI * S * *1 * * * 4 * * *0
Fri *7 May * SI * *LRS *WC *SI LRS * *1 * * * 4 * * *0
Sat *8 May * LR * * LR *LR *LR *LR * *4 * * * 0 * * *4
Sun *9 May * WC * * SI *SI *GC *LR * *0 * * * 3 * * *0 !
Mon 10 May * SI * * *S *SI *SI *WC * *2 * * * 3 * * *0
Tue 11 May * SI* * *SI *SI LRS * D * *2 * * * 2 * * *2
Wed 12 May *LRS * *LRS * S *SI LRS * *2 * * * 3 * * *1
Thu 13 May *LRS* * *SI *SI * S * S * *0 * * * 2 * * *0 !
Fri 14 May * WC * * SI *SI HRS *GC * *0 * * * 3 * * *0 !
Sat 15 May *LRS * * SI *SI *SI LRS * *1 * * * 2 * * *0
Sun 16 May * LR * * LR LRS *SI *SI * *1 * * * 2 * * *1
Mon 17 May *LRS * *LRS * S *SI *SI * *1 * * * 2 * * *1
Tue 18 May * SI * * *S LRS *SI *SI * *2 * * * 3 * * *0
Wed 19 May * SI* * *SI *LR *LR LRS * *1 * * * 2 * * *1
Thu 20 May * SI* * *GC *SI *SI *SI * *3 * * * 2 * * *0
Fri 21 May * SI* * *SI * S *SI *SI * *3 * * * 2 * * *1
Sat 22 May * *S * * SI * S * S *SI * *2 * * * 3 * * *0
Sun 23 May * *S* * * S * S * S * S * *4 * * * 0 * * *4
Mon 24 May * *S* * *SI * S * S *SI * *2 * * * 3 * * *0
Tue 25 May * WC * * SI *SI *SI *GC * *0 * * * 2 * * *0 !
Wed 26 May * SI * * SI *WC *SI *SI * *3 * * * 2 * * *1
Thu 27 May * SI* * *SI *SI *SI LRS * *3 * * * 1 * * *3
Fri 28 May *LRS * * SI *SI *SI LRS * *1 * * * 2 * * *0
Sat 29 May * HR * * LR *LR *HR * S * *1 * * * 3 * * *0
Sun 30 May * SI * * SI *SI *SI *SI * *4 * * * 0 * * *4

For last 30 days: avg 2.13 different forecasts per day
On average forecasted 1.00 days ahead
Correct 4 days ahead: 3 times
Correct 3 days ahead: 4 times
Correct 2 days ahead: 8 times
Correct 1 day ahead: 15 times (50.0%)
Number of times the weather was same as yesterday(*): 11 times (36.7%)
Number of days every forecast was wrong(!): 5 times (16.7%)

Key:
* D: * * * * * * Drizzle *0 occurrences, predicted * 1 times over-forecast
*GC: * * * * *Grey Cloud *0 occurrences, predicted * 4 times over-forecast
*HR: * * * * *Heavy Rain *1 occurrences, predicted * 3 times about right
HRS: * Heavy Rain Shower *1 occurrences, predicted * 3 times about right
*LR: * * * * *Light Rain *3 occurrences, predicted *11 times about right
LRS: * Light Rain Shower *5 occurrences, predicted *14 times under-forecast
* S: * * * * * * * Sunny *3 occurrences, predicted *21 times over-forecast
*SI: * * Sunny Intervals 13 occurrences, predicted *58 times about right
*WC: * * * * White Cloud *4 occurrences, predicted * 5 times under-forecast

-----------------
If it doesn't appear in straight columns, you should view it with a fixed width font.

Now to explain. There are 5 forecasts for each date. D-1 means the forecast made
on the preceding day. So for example, on May 18 the forecast made on that day
for the weather was Sunny Intervals. The previous day (D-1) the forecast for
May 18 had been Sunny and the day before that (i.e. on May 16) the forecast for
two days hence had been Light Rain Showers.
I've assumed that a 5 day forecast is correct if, on day D-4, it predicts the
weather on a particular day AND that the forecast doesn't change in the intermediate
days. As you can see, this hardly ever happens :-( and a random guess wouldn't
be any worse. (In fact if the forecasts said "Sunny Intervals" *every day
they'd be better than the ones produced, right up to the day before).
It's also noteworthy that the accuracy seems to decrease by about 50% each day:
15 - 8 - 4 - 3, given the limits of integer counting.

It seems to me that the basic problem with the forecasts is that they try for too
much detail, too far ahead. So while the "on the day" forecast is probably right-ish
to try to say what the weather will be in 4 days time, to the same level of detail
doesn't work. The understanding and even chaotic elements don't allow it. If you
use less discriminating terms, such as just trying to say if it will be wet or dry
the accuracy goes up quite a lot, Thus:

(excerpted)
* * * * * forecast *forecast on: * *times * num of *days
-- date -- on day * D-1 D-2 D-3 D-4 correct changes ahead
*...
Mon 10 May *Dry* * Dry Dry Dry Dry * *4 * * * 0 * * *4
Tue 11 May *Dry* * Dry Dry Wet Wet * *2 * * * 1 * * *2
Wed 12 May *Wet * *Wet Dry Dry Wet * *2 * * * 2 * * *1
*...

For last 30 days: avg 0.73 different forecasts per day
On average forecasted 2.73 days ahead
* * * * * * * * * * * ^^^^
Correct 4 days ahead: 15 times
Correct 3 days ahead: 19 times
Correct 2 days ahead: 22 times
Correct 1 day ahead: 26 times (86.7%)
Number of times the weather was same as yesterday(*): 21 times (70.0%)
Number of days every forecast was wrong(!): 1 times (3.3%)

Even so. it seems that 2 - 3 days is about all it's possible to forecast
in England.

Well done Pete. A very interesting analysis. You can see some of the
dificulties I have in forecasting at 10 days! It is all about the
level of detail you expect from a forecast, of course.

On Mon, 31 May 2010 09:39:41 -0700 (PDT), Dawlish wrote:

For last 30 days: avg 0.73 different forecasts per day
On average forecasted 2.73 days ahead
Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* Â* ^^^^
Correct 4 days ahead: 15 times
Correct 3 days ahead: 19 times
Correct 2 days ahead: 22 times
Correct 1 day ahead: 26 times (86.7%)
Number of times the weather was same as yesterday(*): 21 times (70.0%)
Number of days every forecast was wrong(!): 1 times (3.3%)

Even so. it seems that 2 - 3 days is about all it's possible to forecast
in England.

Well done Pete. A very interesting analysis. You can see some of the
dificulties I have in forecasting at 10 days! It is all about the
level of detail you expect from a forecast, of course.

Wow, I have no idea how you'd even start about a 10 day forecast. Interestingly
but not shown in my post was that the M.O. are quite good at forecasting
the temperature. They are correct next day 80% of the time, to within 3C.