The Weather Outlook

Remove ads from site

johncs2016
14 February 2022 11:48:29

Those of you will know that when I am reporting any rainfall totals on this forum, I will try to use the data from SEPA (which stands for the Scottish Environment Protection Agency) whenever possible. Indeed, that has always been a useful site for anyone from anywhere in the world who wishes to obtain any rainfall data for anywhere in Scotland, although there is also the backup option of using raw SYNOP/BUFR data from ogimet.com or meteomanz.com if the data from SEPA is unavailable.

Not too long ago, there was a major cyber attack on the SEPA website which meant that the data from there was unavailable for quite a lengthy period of time on a number of occasions. Currently, I am able to get that data from SEPA but since the second occasion when the rainfall data from SEPA was unavailable, there has no longer been any direct link from SEPA's home page to their rainfall data pages.

From the SEPA homepage, you can get to a status page which gives the latest update on how the various sections of the site are operating in the aftermath of those cyber attacks. This status page mentions that the rainfall data pages are available, but there are no actual direct links to them even from there. These rainfall data pages do still exist and you can still access them with any browser, but this can now only be done by manually entering the URL for that into your browser as there are no longer any direct links to them.

However, the status page mentions that a new API system for obtaining rainfall data from there is currently being developed. At the moment, that is in its beta stage which means that it isn't far from being ready for public use and will generally be stable and usable. However, the fact that it is in its beta stage means that it is still unreleased and not quite finished, so there might be more than a few bugs with that.

In the meantime, the old rainfall pages make up the latest released version of that data and is the one to go to for any Scottish rainfall data. However, there will eventually come a time when this new beta system will take over that role and whilst the old system might still be available for a period of time afterwards, there will eventually come a time when that will no longer be the case.

Because of that, I will have to change the way that I go about getting any rainfall data from SEPA for the purpose of reporting that on here, although I should have plenty of time ahead for developing those required changes to my system for doing that. The new system requires you to register for an API key and to me, that is something which should happened a long time ago. This should result in a more secure system in line with how the vast majority of how other weather APIs operate.

The big problem with the old system is that it doesn't have any such requirements for an API key which means that anybody can access any data from there, just with the odd modification to the URL for that. The results in an insecure secure and because of that, it isn't surprising that the old system is open to all sorts of cyber attacks such as what we have seen recently.

If anyone is interested in finding out more about that new API for getting data from SEPA, here is a link to the pages for that.


The north of Edinburgh, usually always missing out on snow events which occur not just within the rest of Scotland or the UK, but also within the rest of Edinburgh.

Crepuscular Ray
14 February 2022 16:00:49
Thanks John. It'll be good to have access to these figures
Jerry

Edinburgh, in the frost hollow below Blackford Hill

johncs2016
14 February 2022 16:27:07

Thanks John. It'll be good to have access to these figures

Originally Posted by: Crepuscular Ray 

It turns out actually that you don't need to request an API key for that new system as long as you are not requesting massive amounts of data at once (such as the all of the last 10 years of data for every single station). This means that I should be able to build a new system out of that in a similar manner to how I developed the old systems for that.

In the meantime, I am still working out how to get any actual rainfall totals from that, but I have been playing around with this system to import data from that into Google Sheets and so far, I have been able to get a list of available rainfall data for Edinburgh Gogarbank and the botanic gardens in Edinburgh (the two stations which I normally report rainfall for).

I will also keep playing around with that though because with this new system in addition to rainfall data, you can also get other data such as river and ground water level data which will come in particularly handy when we're going through periods such as what we had during January when we are getting very little in the way of actual rainfall. That way, I will have a better understanding of whether or not we are actually heading towards a drought scenario under those conditions and as a result, whether or not we should be concerned about that.

 


The north of Edinburgh, usually always missing out on snow events which occur not just within the rest of Scotland or the UK, but also within the rest of Edinburgh.

Remove ads from site