I last wrote about open rail data in December after the ORR’s excellent transparency conference. Many organisations in the rail industry spoke about their desire for better sharing of travel information. They’re great folks and it was mostly a positive affair.
But the bogeyman of the event was ATOC – the Association of Train Operating Companies. They look a bit like a public body but aren’t, instead taking funding from the various train operators. ATOC are fond of making the claim that they don’t take a penny of public money but it’s somewhat disingenuous: their funding comes from train operators and many of those have been subsidized by your taxes.
Through its subsidiary TISL (“Train Information Services Limited”, more commonly known as NRE / National Rail Enquiries) ATOC owns a datasource called Darwin. If you want an API for querying live train running information complete with all the delays and cancellations at any given moment they’re the only shop in town.
Problem is they’ve used their ownership of Darwin in rather an underhand way. Back in 2010 as a punishment for criticising them on this blog they banned me from using the data. There are also significant questions about the fairness of the charges they impose on app developers when they themselves pay no per-app fee. At present if you want to write a smartphone app to display live UK train times you’ll end up giving NRE a chunk of the proceeds or they might just flat-out refuse to give you the data. So, when NRE’s chief exec Chris Scoggins took the stage at the transparency conference developers leapt at the chance to lambast him about the shady way they’ve used their licensing system…
Developer Engagement with ATOC
Getting pilloried like must have alarmed ATOC, and all the more so because it happened in front of the managers of several train operating companies who pay their wages. TOCs want to get people on their trains: pissing off an army of volunteer developers who want to help can hardly be in their interests. Plus with the Office of Rail Regulation’s ongoing consultation on real time train data they can’t risk looking bad.
Eager to give the impression of action NRE organised a three-hour Developer Engagement Day on January 18th at its (and ATOC’s) headquarters in London.
Invitees awoke that morning to the first heavy snowfall of the year. Many were deterred but a few of us still made it. There was tea and biscuits. Chris Scoggins and Derek Parlour made a presentation on how NRE functions and how Darwin is developed. The development is mostly outsourced and the technical architecture developed to pull in live running data from a variety of sources is far from trivial.
NRE say developing and maintaining the Darwin system costs them about £2m per year. I imagine half a dozen open source hackers could do the job for rather less if they had access to the same contacts.
Chris & Derek made much about NRE’s desire to work with developers and expressed a desire to let bygones be bygones. Chris said they’d like to be able to discuss issues with us before they get aired publicly and hinted that I should have talked to them before discussing their behaviour with the press back in 2010. This was weird – I did speak to them first and it wasn’t until they’d said I had little chance of getting a license that I went public. But they seemed to have forgotten that and come up with a more convenient version of history instead.
It went amicably enough and my question at the end – predictably – was “so now can I have access to the data?”
The Darwin License
NRE got in touch a few days later with a copy of the license Darwin users are required to accept.
After what happened in 2010 I gave it a thorough read. One interesting part – but really a red herring – is the commitment to arbitration if a license isn’t granted. It sounds good but has little effect: an arbitrator can only say whether NRE have followed the code they’ve written for themselves, not whether that code is fair.
There are clauses saying you’ll lose access to the data if you bring TISL into “disrepute” and that the signatory may not publicly discuss the subject without written consent from TISL.
15.2. Either party shall be entitled to terminate the Agreement on notice without liability at any time if:
15.3.1. the Customer brings TISL, the TOCs or the passenger rail industry into disrepute
Disrepute is a fabulously broad word – it covers anything that blackens their name even if what you said was true. In other words you aren’t allowed to criticise them.
No public announcement, communication or circular (other than to the extent required by law) concerning the subject matter of this Agreement shall be made or despatched [sic] by the Customer without the prior written consent of TISL.
WTF. You can’t use the live UK train data without accepting a gagging clause.
I queried the clauses and met a brick wall. I did my utmost to be constructive – the app I wrote a few years back is still dead and I’d love to turn it back on. ATOC used the “disrepute” clause to deny me a license in 2010 (Chris put it in writing and I still have his letter) and I certainly didn’t want history to repeat itself.
I attempted to negotiate, suggesting:
- Amending the license for everyone. It’s only one clause and relaxing it would not be controversial.
- Giving me written permission now to blog about my experiences. This would follow the license to the letter and require no changes. Trivial to do and pretty harmless.
I really did try to reach a compromise but after making a halfhearted attempt to defend the gagging rule Chris has stopped answering my emails. The last he replied to was February 1st. It’s now the 22nd. A few developers who attended the engagement day have been trying to get in touch too but I think they’ve met the same wall of silence.
NRE – if you’re reading this – you been blanking me since February 1st. I miss you. All I want is to use the Darwin data without needing your written permission to talk about it. Don’t be a stranger!
Code of Practice
On February 11th ATOC published their new code of practice for real-time systems. A copy of the license is appended. It came as a surprise since I’d expected them to keep the terms a secret. If they hadn’t released it into the public domain I would never have been permitted to blog about the license terms at all.
The Code of Practice is an interesting read. Here are the things that stand out to me:
NRE have so far issued over 140 licences to 3rd parties for use of real time services for various uses including websites, mobile apps and public display screens. We have only ever rejected three applications (one of which we have subsequently agreed a way forward with and the other two were due to the risk of fraud around the proposed usage). [my underlining]
When my application was rejected in 2010 the reason they gave was not risk of fraud so it must the first of the three applications.
In this case the statement is bullshit. At the time of writing no “way forward” has been agreed for the reason detailed above: signing the contract would mean I lost the right to talk about what’s going on and by the time NRE published this I’d already said I had problems with the gagging clauses. The other 140 licensees have already been forced to sign away their freedom of speech but I’d rather hang on to mine. Oh, and it omits to mention that they took two years of ignoring me before this “way forward” was offered at all!
Another point of interest is the push service. It’s basically a firehose of all the UK train data. Rather than you making the odd API query they push new information on train arrivals, departures and delays straight to you. It’s a vital tool for anyone developing a smartphone app that gives immediate alerts to someone on the move.
Push services create greater risk of error by the licensee which could impact on TOC revenues and reputation. Licensees should be aware that they may be required to gives [sic] indemnities and security for damages that any errors they make may cause. Because of this licensees applying for push services should ensure they have the financial capacity to give these indemnities and to put up the security.
What this says is that unless you have some serious resources you can’t have it. Are you a bedroom hacker with a neat idea for alerting people when their train’s running late? Tough, unless you have the “financial capacity” you can’t have the data.
Another point of note is that while developers may have to provide “security for damages that any errors they make may cause” no such liability is accepted by TISL for damages caused to the developer on its own part. Darwin’s data is sometimes wrong; if the developer is expected to bear liability for mistakes at their end why aren’t TISL offering to take liability themselves?
Let’s say I write a smartphone application. I sell it for £5 and £1 each time goes to TISL as a licensing fee. What might happen?
- TISL break their data feed for a week rendering my app useless. My thousand furious customers demand their £5 back but TISL keeps its proportion of the money. Since I have no indemnity against them I lose my profits and reputation plus an extra £1,000 and die penniless in a gutter.
- TISL’s data has regular inconsistencies which annoy my customers. I tell them they missed their trains because TISL’s data was wonky. TISL bans me because I brought them into “disrepute”. My thousand furious customers demand their £5 back but TISL keeps its proportion of the money. Since I have no indemnity against them I lose my profits and reputation plus an extra £1,000 and die penniless in a gutter.
- Some upstream provider of the data feels I do not really have the right to distribute it and sues me. TISL insists that I defend the claim (which it has the right to do) and I rack up huge legal bills. They then exercise their right to make me pay half of them. I am bankrupted. I lose my profits and reputation plus huge legal costs and die penniless in a gutter.
As it stands this rule will prevent any small app developer touching the push data. The traveling public suffers.
What’s Really Going On?
NRE have a stranglehold on the only comprehensive source of real-time passenger train data in the UK. They developed it to power their own website but have recognised in it a way to exert power over anyone who wants to use real time train data and to stifle criticism.
After all the negative press when they locked down the Darwin feed in 2010 and public embarrassment from the audience at the Office of Rail Regulation’s transparency conference last December NRE are trying to give the impression of greater openness. The developer engagement day was designed to look good but from the perspective of a developer it was a sham. The developers who attended that day have not been “engaged” with – instead our emails go ignored and requests for real transparency are rebuffed.
Perhaps the worst thing of all is the artificial barriers to entry which have been created for small application developers needing access to the push data. They make it virtually impossible for any small player to to write a smartphone app that completes with NRE’s own. As smartphone ownership rises this brings NRE a growing revenue stream and they aren’t inclined to share it.
The only way to control this gorilla is to take its power away. Ever since 2010 they’ve used Darwin licensing to stifle dissent and limit innovation. Despite an eagerness to look like they’re opening up nothing has changed. Darwin needs to pass to some publicly owned body – for example Network Rail – who will operate it on behalf of the public rather than its own self-serving interests.
The Office of Rail Regulation’s consultation on Real Time Train Information runs until February 28th so you still have time to get a response in. If you feel that TISL / National Rail Enquiries / ATOC can’t be trusted to steward what should be public data please, please take the time to respond.