Hi

As you are all probably too well aware I’ve recently acquired the Global Surface Archive (SYNOP and METAR) data from Weather Graphics in the United States. It holds data back to January 1930, but really the archive is only solid from the 1st of January 1973. Still this is a major accomplishment as far as I am concerned by Weather Graphics, it represents excellent value for money at just $195 for the standard version. I have had a few problems reading the DVD’s (and there are 11 in the archive), but I seemed to find the washing them in soft hand soap made them readable!

I was just thinking about how many observations the archives contained? So I decided to see if I could calculate an approximate figure.

For main synoptic hours I think a good average figure for land and sea reports is 5,500, for intermediate hours around 2,500 and for other times it drops to around 100. So if you multiply that lot up it comes to

5500 x 4 x 365 x 40 = 321,200,000
2500 x 4 x 365 x 40 = 146,000,000
100 x 16 x 365 x 40 = 23,360,000

So that comes to a staggering 490,560,000 SYNOPs. If you add to this the number of METARs the archive contains (and I know the hourly METAR are usually a duplication of the SYNOP at a lot of sites)

4000 x 4 x 365 x 40 = 233,600,000
2500 x 4 x 365 x 40 = 146,000,000
1700 x 16 x 365 x 40 = 397,120,000

That comes to around 776,720,000 METAR reports, and If you combine them both together, my rough estimate is that the archives contain around 1,267,280,000 individual observations from the last 40 years!

I know the WetterZentrale archives are a great resource for looking at past situations (and I mean a lot earlier than 1973), but I for one, think that to get the true flavour of any weather situation to see the underlying plotted surface observations that the analysis was based on is essential.

I would love to see observations for the Winters of 1947 and 1963, and with that in mind I’ve recently contacted the Met Office to see what it would cost for them to provide me with additional archived data before 1973, I’m sure this would be hugely expensive, but if you don’t ask you don’t find out, and if Weather Graphics can do it why can’t the UKMO? I’m sure the SYNOP format would cover the year 1963, I think the format may have started in the late fifties, but who knows what earlier observations were coded in, and if they have even been ‘digitised’ – if that’s what the correct phrase.

Keep watching my blog for more interesting situations from the archives, and if you have any interesting days from the last 40 years that you would like to see a plotted chart for, please let me know.

Bruce.

On Thursday, October 17, 2013 11:49:27 AM UTC+1, exmetman wrote:
Hi



As you are all probably too well aware I’ve recently acquired the Global Surface Archive (SYNOP and METAR) data from Weather Graphics in the United States. It holds data back to January 1930, but really the archive is only solid from the 1st of January 1973. Still this is a major accomplishment as far as I am concerned by Weather Graphics, it represents excellent value for money at just $195 for the standard version. I have had a few problems reading the DVD’s (and there are 11 in the archive), but I seemed to find the washing them in soft hand soap made them readable!



I was just thinking about how many observations the archives contained? So I decided to see if I could calculate an approximate figure.



For main synoptic hours I think a good average figure for land and sea reports is 5,500, for intermediate hours around 2,500 and for other times it drops to around 100. So if you multiply that lot up it comes to



5500 x 4 x 365 x 40 = 321,200,000

2500 x 4 x 365 x 40 = 146,000,000

100 x 16 x 365 x 40 = 23,360,000



So that comes to a staggering 490,560,000 SYNOPs. If you add to this the number of METARs the archive contains (and I know the hourly METAR are usually a duplication of the SYNOP at a lot of sites)



4000 x 4 x 365 x 40 = 233,600,000

2500 x 4 x 365 x 40 = 146,000,000

1700 x 16 x 365 x 40 = 397,120,000



That comes to around 776,720,000 METAR reports, and If you combine them both together, my rough estimate is that the archives contain around 1,267,280,000 individual observations from the last 40 years!



I know the WetterZentrale archives are a great resource for looking at past situations (and I mean a lot earlier than 1973), but I for one, think that to get the true flavour of any weather situation to see the underlying plotted surface observations that the analysis was based on is essential.



I would love to see observations for the Winters of 1947 and 1963, and with that in mind I’ve recently contacted the Met Office to see what it would cost for them to provide me with additional archived data before 1973, I’m sure this would be hugely expensive, but if you don’t ask you don’t find out, and if Weather Graphics can do it why can’t the UKMO? I’m sure the SYNOP format would cover the year 1963, I think the format may have started in the late fifties, but who knows what earlier observations were coded in, and if they have even been ‘digitised’ – if that’s what the correct phrase.



Keep watching my blog for more interesting situations from the archives, and if you have any interesting days from the last 40 years that you would like to see a plotted chart for, please let me know.



Bruce.

I don't want to be a killjoy Bruce, but I hope the data have all been quality controlled. I know I have checked some daily obs for Plymouth Mount Batten (UKMO) against those available from GSOD on the NOAA site and been a bit alarmed at the differences on some occasions.

Len
Wembury

Len

I'm not exactly certain where Weather Graphics get their SYNOP data but I know it was a massive job for them to retrieve and process it.

I've spent the last 15 years downloading the same kind of data from the GTS and published on the Florida State University web site, and when comparing this with the data in the archives I don't see any real differences. He's done a wonderful job updating old format SYNOPs to new the newer format, and his stations database has the stations that are now closed included. I can even see obs I did back in 1983 at Binbrook - I had a penchant for using 04FU as present weather, whilst a lot of other observers would be reporting the "posher" 05HZ (the trouble with reporting 05HZ at places like Leeming, was that it made it a bit difficult to report 800 M when the sea breeze kicked in from 030° with a RH of 87%. I don't know if they still have the same issue there with Teeside smoke).

I can spot a rogue observations a mile away - I've been zapping duff ships for for so long now that I even know their call signs - you get to know most of the idiosyncrasies for observing sites in Europe and North America, but to be fair they are usually excellent.

In fact North America is a good case in point when you're talking about QC'ing, more often than not you get a mix of current and old SYNOPs, and it can take a while to sort out the old from the new and get a reasonably contoured chart. It's always been an issue in that part of the world, and I don't know what causes it, but the data from the archives has the same problem as the data I get from FSU so I know its what put out on the GTS.

The other thing is that I trust the guy behind Weather Graphics - I've always be more than a little envious with what he has done with Digital Atmosphere and wish that I had pushed myself to get software I had written out there like he has.

Have a look at his site:
http://www.weathergraphics.com/archive/gsa/index.shtml

Bruce.

exmetman wrote:

Len

I'm not exactly certain where Weather Graphics get their SYNOP data but I
know it was a massive job for them to retrieve and process it.

I've spent the last 15 years downloading the same kind of data from the GTS
and published on the Florida State University web site, and when comparing
this with the data in the archives I don't see any real differences. He's
done a wonderful job updating old format SYNOPs to new the newer format, and
his stations database has the stations that are now closed included. I can
even see obs I did back in 1983 at Binbrook - I had a penchant for using 04FU
as present weather, whilst a lot of other observers would be reporting the
"posher" 05HZ (the trouble with reporting 05HZ at places like Leeming, was
that it made it a bit difficult to report 800 M when the sea breeze kicked in
from 030° with a RH of 87%. I don't know if they still have the same issue
there with Teeside smoke).

I can spot a rogue observations a mile away - I've been zapping duff ships
for for so long now that I even know their call signs - you get to know most
of the idiosyncrasies for observing sites in Europe and North America, but to
be fair they are usually excellent.

In fact North America is a good case in point when you're talking about
QC'ing, more often than not you get a mix of current and old SYNOPs, and it
can take a while to sort out the old from the new and get a reasonably
contoured chart. It's always been an issue in that part of the world, and I
don't know what causes it, but the data from the archives has the same
problem as the data I get from FSU so I know its what put out on the GTS.

The other thing is that I trust the guy behind Weather Graphics - I've always
be more than a little envious with what he has done with Digital Atmosphere
and wish that I had pushed myself to get software I had written out there
like he has.

Have a look at his site:
http://www.weathergraphics.com/archive/gsa/index.shtml

Bruce.


I've bought the Pro version which includes time-series of all obs from any
single site for the whole length of the record. Addictive stuff!

By the way, Bruce, I didn't get stung for any VAT or customs charges. As I'm
VAT registered it wouldn't have mattered anyway but to keep the books straight
I'll account for it on both sides of my next VAT return.

--
Norman Lynagh
Tideswell, Derbyshire
303m a.s.l.

On Thursday, 17 October 2013 12:41:59 UTC+1, Len Wood wrote:
On Thursday, October 17, 2013 11:49:27 AM UTC+1, exmetman wrote:

Hi







As you are all probably too well aware I’ve recently acquired the Global Surface Archive (SYNOP and METAR) data from Weather Graphics in the United States. It holds data back to January 1930, but really the archive is only solid from the 1st of January 1973. Still this is a major accomplishment as far as I am concerned by Weather Graphics, it represents excellent value for money at just $195 for the standard version. I have had a few problems reading the DVD’s (and there are 11 in the archive), but I seemed to find the washing them in soft hand soap made them readable!







I was just thinking about how many observations the archives contained? So I decided to see if I could calculate an approximate figure.







For main synoptic hours I think a good average figure for land and sea reports is 5,500, for intermediate hours around 2,500 and for other times it drops to around 100. So if you multiply that lot up it comes to







5500 x 4 x 365 x 40 = 321,200,000



2500 x 4 x 365 x 40 = 146,000,000



100 x 16 x 365 x 40 = 23,360,000







So that comes to a staggering 490,560,000 SYNOPs. If you add to this the number of METARs the archive contains (and I know the hourly METAR are usually a duplication of the SYNOP at a lot of sites)







4000 x 4 x 365 x 40 = 233,600,000



2500 x 4 x 365 x 40 = 146,000,000



1700 x 16 x 365 x 40 = 397,120,000







That comes to around 776,720,000 METAR reports, and If you combine them both together, my rough estimate is that the archives contain around 1,267,280,000 individual observations from the last 40 years!







I know the WetterZentrale archives are a great resource for looking at past situations (and I mean a lot earlier than 1973), but I for one, think that to get the true flavour of any weather situation to see the underlying plotted surface observations that the analysis was based on is essential.







I would love to see observations for the Winters of 1947 and 1963, and with that in mind I’ve recently contacted the Met Office to see what it would cost for them to provide me with additional archived data before 1973, I’m sure this would be hugely expensive, but if you don’t ask you don’t find out, and if Weather Graphics can do it why can’t the UKMO? I’m sure the SYNOP format would cover the year 1963, I think the format may have started in the late fifties, but who knows what earlier observations were coded in, and if they have even been ‘digitised’ – if that’s what the correct phrase.







Keep watching my blog for more interesting situations from the archives, and if you have any interesting days from the last 40 years that you would like to see a plotted chart for, please let me know.







Bruce.



I don't want to be a killjoy Bruce, but I hope the data have all been quality controlled. I know I have checked some daily obs for Plymouth Mount Batten (UKMO) against those available from GSOD on the NOAA site and been a bit alarmed at the differences on some occasions.



Len

Wembury

I bet NASA was up

On Thursday, 17 October 2013 12:41:59 UTC+1, Len Wood wrote:
On Thursday, October 17, 2013 11:49:27 AM UTC+1, exmetman wrote:

Keep watching my blog for more interesting situations from the archives, and if you have any interesting days from the last 40 years that you would like to see a plotted chart for, please let me know.

Got a link, dopey?

I don't want to be a killjoy Bruce, but I hope the data have all been quality controlled. I know I have checked some daily obs for Plymouth Mount Batten (UKMO) against those available from GSOD on the NOAA site and been a bit alarmed at the differences on some occasions.

I sounds like you have found the interesting ones.

Anything anomalous in ANY way at all should be studied meticulously. (Unless you believe in butterflies.)