We’ve been working on a project for WorldSkills Abu Dhabi 2017 since the beginning of the year. You can keep track of our progress on the development blog at http://worldskills.github.io/worldskillsabudhabi2017.com/blog/
We are creating social objects from a neighbourhood’s existing infrastructure which citizens can converse via SMS, Twitter, Telegram and other messaging services to “get in the know” about a specific place.
Our approach goes beyond conventional approaches to open data such as data visualisation, maps, etc and towards humanised suggestions that prompt and guide individual action. The social object speaks in the first person, answering questions as part of a dialogue in a different context for journalism, ie next to friends and other services in SMS and chat apps.
The main questions are:
How can we make the city more searchable and query-able for citizens to help them make the best decisions in their day-to-day lives?
How can a news organisation become part of the fabric of the city?
How can we build a service with a high standard of ethics and integrity, especially with regard to personal data.
How can we make money?
The two key challenges are acquisition and retention. Hyperlocal news is helpful because it gives a reason to follow and excuse for the service to remind you it exists once every day or two. The more fun features can be built on top of that core interaction.
The other questions are around the nature of the objects – single or multiple use, spoke only when spoken to or listening and push out? And how do we make people aware of the services available? This is where being embedded in a city will help – stickers, posters, etc – and having a media company on board – adverts, promos, etc.
A general-purpose Twitter bot that responds to a variety of commands. The main problem here is how to let people know about the range of possible commands available.
A single-purpose Twitter bot that tells you about where to get a burger in the city centre.
https://twitter.com/piccadillywall A Twitter bot around service discovery specifically burger restaurants. Responds to questions around ‘burger nearest’, ‘burger cheapest’, ‘burger best’, ‘burger cleanest’.
The big issue here for me is https://github.com/robertocarroll/chatuitwitterburger/issues/1 Hubot needs a trigger word followed by a modifier. How can we get it to understand more natural questions?
Chat apps such as WhatsApp, Telegram, WeChat and Viber are becoming more popular than social networks such as Twitter, LinkedIn, Facebook, and Instagram. News organisations need to work out ways to deliver valuable, important, original journalism through this new medium. But it’s not just about going where people are. At the heart of our product is the idea of journalism as a conversation. Rather than simply driving traffic to a news organisation’s website, we aim to provide as much information as possible as part of a conversation inside the messaging app. That means focusing on the important local questions people want news organisations to answer to make their lives better. It also means getting beyond the traditional article as the atomic unit of news towards structured data and dialogue.
Designing a trusted automated companion with an appropriate personality to deliver news and information will be a big challenge but also a great innovation if successful.
We will focus on local news and information which would be interesting and useful to people living and working within the neighbourhood of the social object. The most obvious questions are around daily information (weather, transport, etc), hyperlocal news, service discovery (restaurants, cinema, etc) and civic information (crime, schools, planning applications, etc). We’ve started exploring these areas for the initial prototypes by answering the direct questions of citizens.
Another area we are starting to explore is the social object as a nexus for local news and information pulling in local tweets, images, mentions, etc, analysing them and publishing a summary based on that information. For example, we are collecting all tweets with “I feel” or “I am feeling” within the area of the ArndalePostbox, doing sentiment analysis and considering how to use this information. In this way, the social object becomes a witness to history, building up a unique archive of media around a particular area which would otherwise be lost behind rate-limited APIs.
Another area to explore is how we can encourage sharing and encounters in the area. How can we catalyse relationships between people sharing the same physical space and time and then get out the way? The conversational medium makes it natural for people to respond. We’d like to design and develop a service that provides ways for citizens to talk about their neighbourhood and perhaps even collaborate on complex decisions such a planning applications without leaving their messaging app of choice.
The main technical innovation and challenge is to build a coherent, accessible and consistent service on top of existing services and without expensive hardware. There isn’t an app, user logins or even much of a website – everything runs on top of other services through a conversational interface. The chat engine itself is quite primitive at the moment. So far we’ve used the tone and structure of the conversation to develop the experience. It doesn’t involve complex natural language processing nor does it detect and account for context changes in user inputs. We are starting to develop an approach to handling errors and null responses appropriately, preferring to be “as smart as a puppy” rather than trying to respond intelligently to everything. These are points we’d like to explore and develop further, perhaps allowing users to have extended conversations, as well as customise the service to suit their needs over time.
Another innovation will be to create the illusion of a connected object without the usual dependence on electricity, radio, etc. By adopting existing objects in the city, we can bind the experience to a physical object with a location, providing users with a mental model and fixed location for the interaction. We can get many of the benefits of a connected object without the associated issues and expense.
The product will likely consist of:
Conversations answering questions people have about the city as well as ways to gather and collate information from people
A chat system to manage these conversations, including ways to see conversations and monitor metrics such as number of conversations, unique users, etc.
Adapters for various messenger apps
A collection and analysis system to gather data such as local tweets, images, etc.
A push notification system to send out this information to users
A way for users to personalise what they receive
Ways to make people aware of the service and what it can do, e.g. posters in the city, adverts in the paper, etc.
Previous approaches to providing physical information points in a city revolve around kiosks with touch screens. These are expensive and are proven to be under-utilised. If you create a mobile app for local news and information, a user has download it, limiting the audience and keeping the entire experience tied to the screen. We prefer to build on top of existing infrastructure and technologies, stitching together existing services with established communities to create valuable new services in the city.
After exploring the possibilities of SMS, we moved to Twitter for the next stage of the project. This decision was mainly driven by cost: MEN would have to pay for Twilio (the service which powers the SMS functionality) and it would be difficult to recoup that money or justify spending it in the first place.
As it turned out, Twitter was an excellent medium for this kind of interaction. The character count enforced focus, while the idea of an inanimate object in the city sending a Twitter post somehow seemed more convincing than an SMS. The activity was embedded in an object in the city which spoke in the first person, expressed itself through Twitter and even had a glimmer of personality.
The focus of this sprint was being useful, which involved answering as many possible questions as possible and then testing them with users to see which ones they actually did find useful.
Is it going to rain?
We started off with the weather. Rather than churning out a general weather forecast we decided to try to answer the question, “Is it going to rain in the next hour?” We took the detailed forecast from forecast.io, examined the probability and intensity of rain in the next hour and used it to answer the question. It worked quite well:
What’s the news around here?
The news also worked well on Twitter thanks to the cards display.
The medium exposed problems in the content itself: it wasn’t specific enough or regular enough to power this system satisfactorily. The MEN is taking on a city centre blogger, so there will be more relevant content in the future, but the conversational UI does enforce the idea of specifically answering user needs – it’s a question / answer format after all. For this to work well, the content would need to be structured around specific locations and taxonomies. If it was tagged by things like type or mood, we could start to experiment with allowing the user to customise the service by replying to the Twitter post, which opens up all sorts of interesting possibilities.
What’s the most popular news around here?
We used the Chartbeat API to get the local article which is being shared the most on Facebook, Twitter, etc. It worked quite well, although the wording “hottest” story needs to be changed to accommodate all possible stories.
The API at http://data.police.uk has lots of information about specific crimes as well as more general statistics. One option is to show specific crimes at a particular location along with their outcomes. We combined the details such as location, type of crime and the outcome for a single incident into a sentence. Tweet “random crime” and the postbox will reply with one of thousands of crimes from December 2014 that actually happened nearby.
Each tweet gives you a sense of what crime incidents happen in the area as well as their outcome. We felt the randomness reflects – in limited ways – the nature of crime itself. I urge you to try it for yourself.
The general statistics didn’t work quite so well. The idea was that by comparing statistics such the crime rate and total crimes to neighbouring areas you would get a sense of how safe a place was. The city centre’s small population gave it a crime rate of 84.43 and using the total crimes wasn’t easy to convey in a single tweet. There’s still lots to explore with this data.
The general election was coming up as we were working on this sprint so we decided to try incorporating information about the candidates from YourNextMP. It would have been good to find out more about the candidate for a particular party, but the fragility and weakness of the conversational UI emerged since it’s difficult to inform a user about all the different parties in a single tweet. So we settled for random, giving users a broad sense of which candidates were standing and perhaps exposing people to less-well-known candidates.
It was also difficult to convey the distinction between this is who you should vote for and this is who you could vote for, since the tweet was not a recommendation.
Having an extended conversation was always going to be tricky, but we tried it as a way of gathering interesting subjective information from people about their neighbourhood. We asked questions about the neighbourhood and showed someone else’s answer along with the next question.
There were all sorts of problems here including needing to type “answer” to keep the conversation going, problems around the possibility of rude words from mischievous users and the general clunkiness, but it might be worth exploring in different ways in the future.
The final conversation used information about burger restaurants from an MEN article to provide recommendations of a place to eat. At the moment, the suggestion is random:
You get details about the rating – poor, not bad, OK, good and great – as well as the cost of a burger and directions from the postbox. This service was the most popular by far as well as the easiest for people to understand, so we are going to work on improving it in the next sprint.
It was a great sprint exploring a range of different directions with good reaction from users in general to the concept. Here’s some of the things we learnt:
The conversational UI feels useful and provides something different to other ways of interacting with MEN.
The conversational UI is flexible and transferable – SMS, Twitter, WhatsApp, etc may have different affordances – but creating content in a general conversational format means the engine could be used to run different outlets without too much extra effort.
The conversational UI creates an interesting interface on top of various city APIs which would otherwise be difficult to access. Lots of editorial and data work could be done to create useful and interesting information which answers a specific need, e.g. rain, crime. Other possible sources include school data, census data, transport information and sensor data.
The first person framing could be a good place to explore presenting stories by emotion, e.g. awe-inspiring, emotional, positive and surprising.
There are interesting possibilities for gathering information by mixing automated replies along with routing to a human elsewhere.
Service discovery was by far the most popular function with users. They immediately grasped the idea and thought it was useful. So we are going to create a new Twitter account for the Piccadilly Wall in the city centre and use it to find nearby burger restaurants. People will be able to filter the information, eg cheapest, best, nearest, random, cleanest using the command first followed by modifier, ie ‘burger cheapest’.
We are also going to explore the commercial possibilities. What if a person had to retweet the recommendation to get a discount? The place would get publicity to the person’s followers and it would be unrepeatable, i.e. you need the retweet on your timeline to get the discount.
There were also some technical issues around JSON parsing errors, repeated tweets and character counts which we hope to work on to improve the experience.
Hubot is a “customisable, life-embetterment robot” created by Github. They wrote it to automate a range of activities in their company chat room. It’s open source, written in CoffeeScript on Node.js, can be deployed on Heroku and has lots of external scripts written by users. For all these reasons, we decided to use Hubot as the basis for our project.
We chose the Central Library as an exemplar because it was a recognisable building in the city centre and the MEN had a topic page for it.
We spent a while scraping the articles from the MEN topic page and doing various simple things such as the word count, the latest article and the oldest article. We ended up with something like this:
MEN has written 15,626 words about me in the last 7 months and a human would take about 63 minutes to read them. That’s like reading The Communist Manifesto (11,772 words) which Marx and Engels worked on at the nearby Chetham’s library in the 1840s. The latest article was 2 days ago. The oldest article was 7 months ago.
The idea was to connect the current information to interesting things in the city’s history. It didn’t quite work for a variety of reasons including the fact that Manchester apparently has an oral rather literary tradition making it trickier to find a range of books to reference. An interesting diversion nevertheless.
Moving to SMS
In sprint 2 we changed the adapter from Campfire to Twilio, so someone could interact with the system using SMS.
Local news worked well. The interaction felt natural and useful, providing value beyond any existing service and sitting conceptually somewhere between the web and a newspaper. It surfaces neighbourhood articles which might be lost in other mediums, e.g. web, Twitter which focus on latest.
Weather was OK, but it felt a bit pointless. It needed to be more useful, unique and specific. Other apps do the weather better. We wondered if perhaps a specific question would work, e.g. is it going to rain in the next hour?
A bit of personality worked well such as knowing the time of day and the weather and responding accordingly, but having menus inside the application did not work and felt a bit confusing and wasteful.
Lots of things came out of this work. It’s become quite common now, but thinking through making really is important in this type of project. We discovered practical issues such as no picture messages (Twilio MMS is US/Canada only), the potential cost to the user (could feel like “feeding a metre” for users without text-inclusive plans) and the cost to MEN (at $0.04 to send and $0.0075 to receive an SMS is the interact valuable enough?). We also got used to some of the affordances and qualities of SMS as a material. People don’t usually sit and wait for a reply with a text message. They text and receive texts on the move meaning the initial premise of standing in front of an object could be going against the grain. We also discovered that general information, such as the Central library guided tour, is not useful or compelling enough to work in this context.
A quick chat with Tom Armitage, who worked on Hello Lamppost, helped to clarify some of the important conclusions to be drawn here.
SMS tends to be an intimate space usually reserved for messages from friends. Though companies and services are encroaching, it is still a private space with particular opportunities and issues.
There’s no room for extended, light conversations because it is too expensive and maintaining attention could quickly become a problem. Each message must be useful, dense and meaty.
No room for menus or instructions inside the application. We have to make the first text count by making the user choose options in initial text and move the prompts with instructions outside the application.
The conversation is probably going to follow a shape like this:
In the end, the focus came down to being useful. Sprint 3 would focus on user needs with a playful personality. The system could become a convenient way to access the many APIs and data about the city. We would explore the usual suspects such as neighbourhood news, weather, travel delays and departures and open data such as neighbourhood statistics and census information as well as whether it was a good way to gather information from users.
We also decided to move from SMS to Twitter to avoid the question of cost to the user and the MEN.
This post is part of a series about a project exploring conversational UIs for news and information to help Manchester Evening News become part of the fabric of the city.