Friday, October 4, 2019

Accuracy of temperature predictions

Using the weather data I collected previously, I wanted to figure out which of the forecasts services was making the most accurate forecasts.

The way I calculated this was by comparing the temperature predicted in x days time with the actual temperature reported by the service. e.g. if the provider predicted on 1st/Aug that the temperature would be x degrees in 5 days time, then I compare this with their actual reported temperature on 6th/Aug, This absolute error is then averaged across all readings from  all cities

Here is the summary data

Here is a chart of the data for people who prefer visual:

Points to note
  • I eliminated one reading from weather underground 6 day forecasts because it would have re-scaled the chart (probably indicates a programming error on my behalf).
  • In general the forecast accuracy decreases as the number of days increases.
  • is best for next day, but bluemix and darksky are equally good further out.
  • OpenWeatherMap and Weather underground are significantly worse.

This analysis lumps all cities together for an average result. I might do some analysis later on performance per city.

Weather forecast data

I have published several posts about weather. I also did articles with analysis of temperature and wind forecast, but there is a lot more analysis which could be done. Therefore I decided to publish the raw data in case anyone wants to analyse it. I will also do some more charting/plotting myself.

Click here to download the captured forecasts in CSV format. The file contains over 2 million records in total and the format is fairly self explanatory. It covers a period from June 2017 to December 2018, but some forecasts are missing due to a variety of factors. Most notably the Bluemix data only covers August 2017 to February 2018.

I will later post additional analysis files in this folder with some derived data from this original data in the CSV.

I recently found out about a professional analysis report comparing the accuracy of several forecast providers. I think you should value their results above mine.

Sunday, September 29, 2019

Polite new icon

I was calling someone recently and I watched the screen while waiting to see if they would pick up.  After a few seconds passed I noticed this nice icon appearing on the screen asking for understanding if the person didn't pick up.

I thought it was very cute and encouraged a polite response to someone not answering my call. Maybe it is not new, but it was my first time to notice it.

Thursday, June 13, 2019

More challenges to navigating around dublin

Visitors (and even locals) often find it hard to navigate around Dublin. One of the problems the strange custom in Dublin of assigning totally different names to different sections of the same road.  I blogged before about some of the more confusing examples of this. Another problem is the fact that the council regularly decide to change the name of a street.

Here are a few examples:
  1. Amiens Street was The Strand 
  2. Ardee Street was Crooked Staff 
  3. Aughrim Street was part of Blackhorse Lane 
  4. Back Lane was Rochel Street 
  5. Balfe Lane was Pitt Street 
  6. Belmont Avenue was Coldblow Lane 
  7. Benburb Street was Barrack Street 
  8. Bishop Street was Great Boater Lane 
  9. Blackhall Place was The Gravel Walk 
  10. Bow Lane was Elbow Lane 
  11. Bow Street was Lough Buoy 
  12. Brabazon Street was Cuckold's Row 
  13. Bridgefoot Street was Dirty Lane 
  14. Brookefield Avenue was Watery Lane 
  15. Brookefield Road was Cutthroat Lane 
  16. Camden Street was St. Kevin's Port 
  17. Cathel Brugha Street was Gregg Lane 
  18. Cathedral Lane was Cabbage Garden Lane 
  19. Chancery Place was Mass Lane 
  20. Christchurch Place was Skinners' Row 
  21. Clonliffe Road was Fortick Road 
  22. Collage Green was Hoggen Green 
  23. Constitution Hill was Glasmunogue 
  24. Dame Street was Tengmoutli Street 
  25. Dean Street was Cross Poddle 
  26. Digges Lane was Goat Alley 
  27. Dorset Street was Drumcondra Lane 
  28. Engine Alley was Indian Alley 
  29. Essex Street was Orange Street and earlier it was Smock Alley 
  30. Exchange Street Lower was The Blind Quay 
  31. Exchequer Street was Chequer Street 
  32. Foley Street was Montgomery Street and earlier it was World's End Lane 
  33. Glover's Alley was Rapparee Alley 
  34. Green Street was Abbey Green 
  35. Haddington Road was Cottage Terrace 
  36. Hammond Lane was Hangman's Lane 
  37. Harrington Street was Whitworth Street 
  38. High Street was Main Street 
  39. Hill Street was Lower Temple Street 
  40. Infirmary Road was Aberdeen Street 
  41. Island Street was Dunghill Lane 
  42. Lansdowne Road was Watery Lane 
  43. Lincoln Lane was Pudding Lane 
  44. Lincoln Place was Park Street and earlier it was St. Patrick's Well Lane 
  45. Little Green Street was Bradogue Lane 
  46. Little Ship Street was Pole Mill Street 
  47. Lower Baggot Street was Gallows Road 
  48. Lower Gardiner Street was The Old Rope Walk 
  49. Mary's Lane was Broad Street 
  50. Mercer Street was French Street 
  51. Mespil Road was Gibbet Meadow 
  52. North Brunswick Street was Channel Row 
  53. North Lotts was Newfoundland 
  54. O'Connell Street was Sackville Street 
  55. Upper O'Connell Street was Drogheda Street 
  56. Old Kilmainham was Murdering Lane 
  57. Oliver Bond Street was Mullinahack 
  58. Parnell Square was Rutland Square 
  59. Parnell Street was Great Britain Street 
  60. Pearse Street was Great Brunswick Street 
  61. Pimlico was Donour Street 
  62. Prussia Street was Cabragh Lane 
  63. Railway Street was Upper Tyrone Street and earlier it was a Part of Mecklenburgh Street (see Waterford St.) 
  64. Sackville Place was Tucker's Row 
  65. Sean McDermott Street was Gloucster Street and earlier it was Great Martin's Lane 
  66. Shelbourne Road was Artichoke Road 
  67. St. Andrew's Street was Hog Hill & earlier it was Hoggen Hill 
  68. St. Michael's Lane was Macgillamocholmog's Street 
  69. St. Stephens's Green North was Beaux Walk
  70. St. Stephen's Green South was Leeson's Walk 
  71. St. Stephen's Green East was Monk's Walk 
  72. St. Stephen's Green West was French Walk
  73. Talbot Street was Cope Street North 
  74. Townsend Street was Lazy Hill and earlier it was Lazar's Hill 
  75. Usher's Lane was Dog And Duck Yard 
  76. Waterford Street was Lower Tyrone Street and earlier it was Part of Mecklenburg Street (see Railway Street) 
  77. Wellington Street was Paradise Row 
  78. Wolfe Tone Street was Stafford Street
This is not a complete list of street name changes, but just be careful if you have an older map.

Monday, September 3, 2018

Call for Code Project

As you may have heard, Ginni Rommety the CEO of IBM recently launched a Call-for-Code challenge to build a solution using IBM technology which will help solve problems with responding to natural disasters

We had a one day hackday in the Dublin lab to help form teams and kick-off projects. During this day I became part of a team named "Watson Dispatch Manager" which aimed to build a call screening agent for emergency services. This aims to solve the problem that when a major incident happens the emergency services get overwhelmed by multiple callers reporting the same incident, while callers trying to report a different (equally urgent problem) can't get through.

After the hackday, the team continued working and build and actual working prototype.

Here is a presentation about the project

Here is a video about the project for people who prefer that format
And here is an actual video of the prototype in operation.

Thursday, June 14, 2018

Keeping track of unread marks in Slack

In recent years IBM (and other companies) have become enthusiastic users of Slack to communicate. I must admit that I am not so impressed with Slack (partly because I was involved in developing some alternatives that I consider better), but I am leaning to use it nevertheless.

A confusing thing about Slack is how the concept of Workspaces and Channels intersect. On the far left of the Slack client you see a list of icons showing the Workspaces that you have joined. When you click on a particular Workspace , you will see a list of Channels in the Workspace and when you click on a channel you see the associated message. This sounds very straightforward, but the confusing thing is that you can have lots of Shared Channels and these will appear in multiple workspaces.

Like most messaging systems, Slack allows you to keep track of the messages you have read. I frequently load up Slack and read unread messages in each of the workspaces. However, I notice that when I switch to a second workspace the messages in the shared channel which I marked as read in the first workspace  are still showing up as unread.

Eventually I figured out how to solve this problem. You need to either select the menu item "View\Refresh" or else type ctrl-R each time you switch workspaces. The Slack screen goes blank for about 30 seconds and then comes back with the unread marks correctly applied. It is annoying to have to do this, hopefully Slack with fix this bug soon.

Thursday, March 1, 2018

Un-shortening URLs

URL shortening services such as are very popular, but sometimes they can be dangerous. This is why I am glad to see that a url un-shortening service is now available.

The way url shorteners work is that I can set up as a short url for and then you can save yourself some typing by typing into your address bar and be brought directly to . In this case the number of keystrokes is hardly worth the effort, but in many cases URLs can be very long and hence difficult to type in correctly.

The danger of short URLs is that you don't know where they are going to bring you to. For example you might be expecting to be brought to but instead find yourself brought to

The way works is that you go there and type in your short URL and it tells you where you will be redirected to if you use that short URL (this might involve multiple redirects). You can then decide whether or not you feel safe to visit the site in question.

Thursday, January 11, 2018

My new GlowOrb tells me when rain is imminent

I recently acquired a new gadget called a GlowOrb. This is an orb the size of a ping-pong ball which is MQTT enabled which means that it can be programmed to change colour to give an ambient indication of some metric you would like to be aware of. Previous owners have used their GlowOrbs for a wide variety of uses e.g. alerting on air quality changes. However, I am obsessed with rain (as are almost all Irish people) so I decided to configure my GlowOrb to alert me of the chances of rain in the immediate future.

I followed the colour scheme from official weather warnings. So if my GlowOrb is
  • Green (as in the picture) it means that there is < 20% chance that it will rain in the next 4 hours and I can cycle to work with confidence that I won't be like a drowned rat when I get there.
  • Yellow tells me that the chance of rain is between 20% and 40% so bringing a jacket might be a good precaution
  • Orange indicates that the probability has grown to 40% - 60%
  • Red implies that the chance of rain is over  60% so there is no point in putting out the washing.
The way it works is that each GlowOrb has a serial number (printed on the underside of the unit) and you send a MQTT message containing a colour code to a MQTT channel associate with the serial number to change the colour of your orb. Don't worry if you don't understand what this means, there is a web address like printed on the base of the unit (where XXXX-YYYY is the serial number of your unit) and if you visit this web address you will be given detailed instructions on how to use the GlowOrb. It also gives you a piece of JSON that you can import into any NodeRed service to have a working control program for your GlowOrb.

All I had to do to customise my GlowOrb was to write a simple NodeRed flow which accesses the BlueMix weather service every 15 minutes to determine the likelihood of rain where I live and then change the colour of the GlowOrb to match.

If you can imagine a use for your own GlowOrb, I encourage you to get one and experiment. They are quite cheap, but there is no formal ordering or pricing process for GlowOrbs. However, if you send a twitter message to @AndySc, he will give you a price quote in bitcoin or pound sterling.

Sunday, November 12, 2017

The life and death of the @llweather weather station

It seems that my personal weather station is going to be off line for several months (or maybe forever), so I thought I ought to write a short blog post to explain the history of the station and why I have to take it off line. It is a good example of how much work can be involved in keeping a service running despite the fact that the service is supposed to be completely automatic.

Here is the history:
  • My story started during the cold winter of  2010 in Ireland. The weather was so severe that this cold snap has it's own wikipedia page. As a result, I became very interested in weather and I dropped a strong hint that I wanted a weather station kit from Maplin as my Christmas present.
  • This kit included some software to upload my weather data to the internet. This seemed like a cool idea to me, but the software only worked on Windows and I didn't like the idea of leaving a windows PC running 24/7.
  • I used Google to look if there was other software I could use and found several open source options. Eventually I picked a python library pywws and I staredt publishing to my personal weather underground site in March 2011 with a Tonido Plug  (an Ubuntu Linux based plug computer) doing the hard work.
  • In April/May 2015 my plug computer died a natural death due to overheating. Instead of buying a new plug I replaced it with a Raspberry Pi because that had been released in the meantime and it was the new cool thing to use for hobby projects (it was also smaller and cheaper than a new Tonido plug).
  • As part of setting up the Raspberry Pi, I downloaded a new version of pywws. I noticed that  it now had support for uploading to a number of new services including Twitter and so I established the @llweather account to store my tweets. 
  • I though other geeks might be interested in what I did an so I spoke at 3Dcamp Limerick in May 2015  At this talk, an audience member warned me  that I was lucky to have a station still working because the normal lifespan is 2-3 years. This prediction was proven true when shortly afterwards the indoor weather hub smashed to pieces in an accident Apparently it is normally the outdoor equipment that dies due to the severity of the weather it is monitoring, but due to my own clumsiness my home office was a more severe environment than any Irish storm. It wasn't possible to buy a new hub unit so I bought a completely new kit. Luckily my old software configuration worked fine with the new hardware and service was quickly restored.
  • The software was completely automatic, but I found that sometimes my site stopped sending updates and although it was normally easy to fix the problem, it took me some time to notice that there was an issue. Therefore I implemented a monitoring service based upon tweets or lack of same. Since my employer IBM had recently launched the Bluemix service I implemented this in NodeRed running on Bluemix.
  • The monitoring software would send me a Twitter DM whenever it noticed that the weather station had stopped sending tweets or if it started sending blank tweets. There were several possible causes but I noticed that they were normally solved by tightening the USB cable and/or rebooting the Raspberry Pi. After a while I stopped investigating the cause and just tried simple solution. I only investigated the issue when this failed to solve the issue.
  • In early 2017, the station started warning much more often often. Since these problems often solved themselves, I loosened the rules about when to issue a warning (e.g. 40 minutes without a tweet instead of 25 and ignore the first 2 blank tweets). However. the problem got even worse and I couldn't ignore it anymore. I suspected that the raspberry pi was at fault because it was getting old, but a new replacement was not significantly better. I eventually thought to replace the power supply and problem went away for 1 month.
  • When I investigated. I noticed that the indoor hub was getting either no data or crazy data e.g. 51 degrees centigrade in Ireland. If this was Dubai it could mean hot weather, but in Dublin could only mean equipment malfunction. A quick Google said it was most likely a problem with dampness in transmitter unit. When I disassembled the unit there was indeed signs of water damage. Unfortunately, the recommended solution of leaving it in the hot press for a few days didn't solve the problem so I decided to buy a replacement transmitter.
  • The only supplier of spares is Maplin (or amazon/ebay) but all said that they were sold out. I then went to local Maplin store and they said none anywhere in Ireland. 5 available in English Maplin stores but they couldn't tell which store and it might be a computer error. Their web store won't have any in stock until Feb 2018 and I couldn't place an order now even if I was willing to wait
  • The Maplin guy said that it might be possible to buy a spare from another supplier, but he couldn't tell me which brands were compatible with Maplin N96GY. I found some websites selling sparese for the Nevada WH1080PC weather statio which is allegedly co,patible, but when I found one online their picture looked slightl different to mine so I was reluctant to buy

I eventually decided to retire the station for now and reconsider in Feb if I buy a spare when they become available. Overall I must admit I was surprised at how much work was involved in keeping a relatively simple service online.

Tuesday, September 12, 2017

MQTT presentation for IOT meetup

Tomorrow evening I will be giving a presentation about MQTT at the IOT Dublin meetup. Click here if for further details if you want to attend (several other interesting speakers also).