My role in adding Sentiment Analysis capability to IBM's Smarter Cities offerings

Earlier this year I announced that I was changing jobs again to incubation team leader. In this role I am supposed to lead innovative projects to introduce experimental new features in our products. The nature of these innovations means that there is bound to be some risk of failure, but I am delighted to be able to say that the first project to develop a city sentiment dashboard was an even bigger success than expected. In order to ensure were meeting real customer needs, we developed it in conjunction with a large European City who gave us very specific requirements. The end result was so well received that this week the software was formally upgraded from a customised solution into a fully supported IBM product.

Our solution uses Cognos Consumer Insight (CCI) to analyse sentiment being expressed about the themes that the city is interested in. CCI provides really excellent analysis capabilities, but the user interface can be daunting for non-experts to use.  Hence, the software we developed pulls the key information into a single easy to understand dashboard that is part of the Intelligent Operations Centre (IOC). Our dashboard was released in the form of an update to the IOC. Customers will typically deploy this software as part of a Smarter Cities project. If you are interested you should get in touch with your IBM sales representative or if you don't have an existing relationship with IBM leave a comment here and I will arrange for someone to get in touch.

Since IOC has many features, the information about the new sentiment analysis feature is buried deep on the product page. The screenshot below shows the dashboard we developed. I deliberately blurred the text in the screenshot because I don't think that I am at liberty to reveal what city we were working with or what topics they wanted to track. However, I think you can get a feel for how we managed to squeeze all of the sentiment data onto a single easy to understand screen:

• On the left there is a hierarchical list of the themes that the customer specified that they are interested in tracking sentiment about. For each theme there is a thumbs up or thumbs up icon to give a quick indication of whether we found more positive or negative expressions of sentiment about the theme in question.
• If you click on a theme, we will display details about the sentiment we discovered for this theme. We have pie charts showing the overall distribution of sentiment observed as well as a more detailed breakdown of the sentiment for each type of site (e.g. we might have observed radically different sentiment in newspapers, blogs and twitter).
• On the bottom we have a trend chart showing how the sentiment statistics varied over time and finally on the right hand side we show you a few samples of positive and negative snippets that we found. Clicking on these snippets will bring you directly to the original source of the sentiment so you can read the snippet in context.

It is amazing what can be achieved by a small agile team in 6 months. Hopefully we will be equally successful with our next projects (which have not yet been decided).

Innovation, Invention and Inclubation

I blogged last May about my new job as a "Sametime Consumability Expert" and I wrote another update in October to let people know how I was getting on in this new role. However, eagle eyed readers of this blog will have noticed that the "about me" section was recently updated again and that my new job title is Incubation Team Leader.

The reason I didn't write a blog post about this new role when I found out about it a few months ago not because I was being secretive, but I needed to figure out what the job involved first before I could tell others. I am now settled into the job and my first project has kicked-off so I think I can confidently write a brief overview.

The first thing that you might ask is what do I mean by incubation. The term is quite ambiguous, but in this context incubation is helping newly invented technologies cross the chasm into innovations that are used within IBM's products. IBM is rightly proud of their record on invention since they have been the most prolific filers of patents each year for the last 19 years. However, the real goal is innovation that matters and while IBM does well at innovation, their leadership in innovation is not as clear as it is with innovation. I don't think that my small team will be able to solve all of IBM's innovation needs, but we do hope that we can make a small but significant growth in the amount of innovation in IBM's products that come from the Ireland lab.

In practice what I will be expected to do is to lead a series of teams who will be assembled to complete a proof-of-concept  integration of some promising new technology into IBM's Smarter Planet product set. Each project will be scheduled to last between 3 and 6 months.  The idea of limiting the projects to such short duration is to ensure that I can tackle a large volume of project ideas. In any case if the technology is really as promising as we thought at the start - then 6 months should be enough time to do enough work on the prototype to persuade at least one paying customer to buy it and/or persuade a product team within IBM to formally fund a permanent team.

In theory the team will be open to projects which use inventions from either inside or outside IBM. However, when I established an ideation blog to allow lab staff contribute ideas for the team to incubate I was pleasantly surprised to find that I quickly got enough ideas to keep me busy with a series of teams for the next year or two before we need to look outside for inspiration.

The first project team has just been formed. I can't say exactly what the project involves - partly for company confidentiality reasons, but mainly because the project definition is changing rapidly. All I can say at this stage is that it will enable our Smarter Cities customers get a dashboard with an overview of what the citizens of the city are saying on social media about what they think of the way the city is being administered. The project will be completed (successfully or otherwise) by August so I will be able to reveal all at that stage. :-)

I am confident that this is a job that I will really enjoy and hence be successful.