long article about "free the data" - guardian (UK)

[Michael Lenczner]

Give us back our crown jewels

Our taxes fund the collection of public data - yet we pay again to
access it. Make the data freely available to stimulate innovation,
argue Charles Arthur and Michael Cross

 Thursday March 9, 2006
The Guardian

Imagine you had bought this newspaper for a friend. Imagine you asked
them to tell you what's in the TV listings - and they demanded cash
before they would tell you. Outrageous? Certainly. Yet that is what a
number of government agencies are doing with the data that we, as
taxpayers, pay to have collected on our behalf. You have to pay to get
a useful version of that data. Think of Ordnance Survey's (OS) mapping
data: useful to any business that wanted to provide a service in the
UK, yet out of reach of startup companies without deep pockets.

This situation prevails across a number of government agencies. Its
effects are all bad. It stifles innovation, enterprise and the
creativity that should be the lifeblood of new business. And that is
why Guardian Technology today launches a campaign - Free Our Data. The
aim is simple: to persuade the government to abandon copyright on
essential national data, making it freely available to anyone, while
keeping the crucial task of collecting that data in the hands of
taxpayer-funded agencies.

One government makes the data it collects available free to all: the
United States. It is no accident that it is also the country that has
seen the rise of multiple mapping services (such as Google Maps,
Microsoft's MapPoint and Yahoo Maps) and other services - "mashups" -
that mesh government-generated data with information created by the
companies. The US takes the attitude that data collected using
taxpayers' money should be provided to taxpayers free. And a detailed
study shows that the UK's closed attitude to its data means we lose
out on commercial opportunities, and even hold back scientific
research in fields such as climate change.

Who are the culprits? Besides OS, with its vast and valuable map data,
there are the UK Hydrographic Office (which collects tidal and naval
navigational data), the Highways Agency (which collects traffic data)
and even the European Centre for Medium Range Weather Forecasting.

Britain's public sector information is held by some 400 government
departments, agencies and local authorities. Assets range from wills
dating back to 1858, house values recorded in the Land Registry, maps
and the risk of flooding to individual homes. Much is of great
commercial interest, especially when it can be presented on innovative
websites such as upmystreet.com. These sets of data are the modern
crown jewels - but instead of treating them as a resource to boost
national wealth, the government locks them up, restricting access to
those who pay.

The heart of our argument is twofold. First, the government should not
run businesses. The civil service is too inflexible to cope with the
speed of change in the commercial sector. If a company like Microsoft
finds it difficult to adjust to a world of online applications, how
much harder is it for a government department rooted in paper-based
processes?

Second, the government should be charged with collecting the best
data. The Office for National Statistics labours to collect the most
accurate depiction of Britain's society and economy. No private
organisation has the resources, time or breadth of approach. By
contrast, commercial companies ignore less profitable sectors: we
can't trust data collected by banks about spending habits because they
ignore people who cannot afford to open accounts. Similarly, few
companies make detailed maps of Britain; they concentrate on the areas
with the best returns.

In a seminal piece of research into the real cost of charging for
access to public data, the late Peter Weiss, of the US National
Weather Service, compared open and closed economic models for public
sector data. His paper, Borders in Cyberspace: Conflicting Public
Sector Information Policies and their Economic Impact, is online
(http://tinyurl.com/cby55). He quoted a 2000 study for the European
Commission carried out by Pira International, which noted that "the
concept of commercial companies being able to acquire, at very low
cost, quantities of public sector information and resell it for a
variety of unregulated purposes to make a profit is one that
policymakers in the EU find uncomfortable." But why?

Pira pointed out that the US's approach brings enormous economic
benefits. The US and EU are comparable in size and population; but
while the EU spent €9.5bn (£6.51bn) on gathering public sector data,
and collected €68bn selling and licensing it, the US spent €19bn -
twice as much - and realised €750bn - over 10 times more. Weiss
pointed out: "Governments realise two kinds of financial gain when
they drop charges: higher indirect tax revenue from higher sales of
the products that incorporate the ... information; and higher income
tax revenue and lower social welfare payments from net gains in
employment."

Infinite applications

The applications for public sector data are infinite, but here are two
real-world ones that affect ordinary businesses. Many of Britain's
best rock-climbing venues are on sea cliffs, and hence affected by the
tides. For climbers planning a trip - and surely spending money in
local shops - it helps to know if the tides will be favourable. But
websites that try to offer British tide data have been told by the UK
Hydrographic Office they must pay for it - a cost most are unwilling
to endure. So sites have no tides, climbers make the safer choice, and
shops miss out.

Take traffic data. The Highways Agency has set up an exclusive deal
with TrafficTV to send video details from motorway cameras to your
mobile. But if the data were free, any company could offer
well-organised maps and drive down the price while increasing the
numbers who would have better access to traffic data. That would
reduce congestion and pollution.

How did we get into this mess? The problem stems from the days of the
Conservative government, which created "trading funds". These
government-owned bodies - including OS, Companies House, the Driving
Standards Agency, HM Land Registry, the Patent Office and the Royal
Mint - have to sell to other parts of the government and private
customers. Every year they have a "revenue target" that must be paid
to the Treasury; but the only way to earn those revenues is through
commercial activity - creating the ludicrous spectacle of
government-approved monopolies making the market buy data at their
rates. A growing number of experts say the loss to the economy
outweighs "gains" through lowered taxes. "The amount of money made is
trivial," says Keith Dugmore of the Demographics User Group, which
represents commercial users of government information.

But the stifling of competition is most serious. Sometimes, the
conflicts are surreal. The supply of postal addresses is an example.
Three publicly owned agencies - the Post Office, OS and a consortium
of local authorities - run lists of addresses. Because they are forced
to put a "value" on the lists, they have been unable to agree a way of
creating a single database. The cost of licensing addresses was one
reason why returns from the 2001 census were so poor from several
fast-changing metropolitan areas, such as Manchester.

A local authority such as Swindon has to pay OS £38,000 a year to use
its addresses and geographical data. It also has to pay the Royal Mail
£3,000 for every website that includes the facility for people to look
up their postcodes. Yet it was local authorities, which have a
statutory duty to collect street addresses, that collected much of
this data.

Intense conflict

Kristin Woodland, who chairs the local authorities' street gazetteer
group, says: "The taxpayer pays for us to create the data, then has to
pay us to use the data."

The conflict between a trading fund's traditional role and its new
commercial activities is most intense in mapping. "Technology has left
the licensing model far behind," says Dugmore.

Particular controversy surrounds the role of OS, founded to prepare
the British army for a French invasion and now a household name that
sees itself at the forefront of geographical information systems.

OS has a monopoly on centuries' worth of data thanks to taxpayer
funding. Without that head start, it would be in the same position as
any other startup today: facing the choice of creating a brand-new
dataset, or finding someone who had collected it and licensing it from
them.

Part of the problem is that government - and the Treasury - is blinded
by its internal commercial targets, and unable to see the damage its
policies do. Thus within government, OS is widely seen as an example
of a venerable public body transforming itself into a dynamic
commercial enterprise. Last year it had a surplus of £9.2m, though
only part was returned to the Treasury. But the bulk of its revenues
come from the public sector, partly through a payment called the
National Interest Mapping Services (Nims), worth £13m a year, and
partly from deals with government bodies.

The latest of the deals is under negotiation. Yesterday, the
government was due to receive bids for a £20m contract to provide
geographical information to Whitehall bodies. In theory, it is an open
competition. In practice, the largest part of the contract, for "large
scale data", is unlikely to go to anyone else: the contract specifies
that the winner must have large-scale maps in place within six weeks
of the start of a five-year deal.

The OS told the Guardian: "We are bidding for pan-government agreement
and we operate in fair competition under heavy scrutiny. The only
monopoly we have is in the mapping of uneconomic areas." In 2004-05,
47.5% of its £100.4m trading revenue came from government (which
allows it to claim that it is not government funded). It says Nims is
not a subsidy: "This is a contribution to the cost of mapping
uneconomic areas."

Profit centres

That makes sense - if you consider the geography of the United Kingdom
to be a profit centre. But if you view it instead as belonging to its
taxpayers, and meriting rigorous mapping for their benefit, there are
no "uneconomic areas" - only places that people haven't started to use
yet.

Happily, the practice of state-owned monopolies competing in markets
dependent on their information is under attack from several quarters.
A new trade association, Locus, is calling for the government to
enforce a level playing field in the market in public sector
information.

And some of the most copyright-mad government organisations have
relented. Hansard, the record of proceedings in parliament, was once a
paper product that had to be bought from Her Majesty's Stationery
Office. Now it is online, for free, so that sites like
theyworkforyou.com can generate useful data about individual MPs'
voting and attendance.

The Office of Fair Trading is preparing a report on public sector
information, due this summer, which will "look at whether or not the
way in which public sector information holders (PSIH) supply
information works well for businesses. It will examine whether PSIHs
have an unfair advantage selling on information in competition with
companies who are reliant on the PSIH for that raw data in the first
place."

Though it may already be shooting for an open goal, we urge the OFT to
compare the UK with the US; read Peter Weiss's paper; and then,
finally, to free our data.

[Hugh McGuire]

so, when do you think the globe & natl post will make their big
announcements? ;)

Michael Lenczner wrote:

> Give us back our crown jewels
>
> Our taxes fund the collection of public data - yet we pay again to
> access it. Make the data freely available to stimulate innovation,
> argue Charles Arthur and Michael Cross
>
>  Thursday March 9, 2006
> The Guardian
>
> Imagine you had bought this newspaper for a friend. Imagine you asked
> them to tell you what's in the TV listings - and they demanded cash
> before they would tell you. Outrageous? Certainly. Yet that is what a
> number of government agencies are doing with the data that we, as
> taxpayers, pay to have collected on our behalf. You have to pay to get
> a useful version of that data. Think of Ordnance Survey's (OS) mapping
> data: useful to any business that wanted to provide a service in the
> UK, yet out of reach of startup companies without deep pockets.
>
> This situation prevails across a number of government agencies. Its
> effects are all bad. It stifles innovation, enterprise and the
> creativity that should be the lifeblood of new business. And that is
> why Guardian Technology today launches a campaign - Free Our Data. The
> aim is simple: to persuade the government to abandon copyright on
> essential national data, making it freely available to anyone, while
> keeping the crucial task of collecting that data in the hands of
> taxpayer-funded agencies.
>
> One government makes the data it collects available free to all: the
> United States. It is no accident that it is also the country that has
> seen the rise of multiple mapping services (such as Google Maps,
> Microsoft's MapPoint and Yahoo Maps) and other services - "mashups" -
> that mesh government-generated data with information created by the
> companies. The US takes the attitude that data collected using
> taxpayers' money should be provided to taxpayers free. And a detailed
> study shows that the UK's closed attitude to its data means we lose
> out on commercial opportunities, and even hold back scientific
> research in fields such as climate change.
>
> Who are the culprits? Besides OS, with its vast and valuable map data,
> there are the UK Hydrographic Office (which collects tidal and naval
> navigational data), the Highways Agency (which collects traffic data)
> and even the European Centre for Medium Range Weather Forecasting.
>
> Britain's public sector information is held by some 400 government
> departments, agencies and local authorities. Assets range from wills
> dating back to 1858, house values recorded in the Land Registry, maps
> and the risk of flooding to individual homes. Much is of great
> commercial interest, especially when it can be presented on innovative
> websites such as upmystreet.com. These sets of data are the modern
> crown jewels - but instead of treating them as a resource to boost
> national wealth, the government locks them up, restricting access to
> those who pay.
>
> The heart of our argument is twofold. First, the government should not
> run businesses. The civil service is too inflexible to cope with the
> speed of change in the commercial sector. If a company like Microsoft
> finds it difficult to adjust to a world of online applications, how
> much harder is it for a government department rooted in paper-based
> processes?
>
> Second, the government should be charged with collecting the best
> data. The Office for National Statistics labours to collect the most
> accurate depiction of Britain's society and economy. No private
> organisation has the resources, time or breadth of approach. By
> contrast, commercial companies ignore less profitable sectors: we
> can't trust data collected by banks about spending habits because they
> ignore people who cannot afford to open accounts. Similarly, few
> companies make detailed maps of Britain; they concentrate on the areas
> with the best returns.
>
> In a seminal piece of research into the real cost of charging for
> access to public data, the late Peter Weiss, of the US National
> Weather Service, compared open and closed economic models for public
> sector data. His paper, Borders in Cyberspace: Conflicting Public
> Sector Information Policies and their Economic Impact, is online
> (http://tinyurl.com/cby55). He quoted a 2000 study for the European
> Commission carried out by Pira International, which noted that "the
> concept of commercial companies being able to acquire, at very low
> cost, quantities of public sector information and resell it for a
> variety of unregulated purposes to make a profit is one that
> policymakers in the EU find uncomfortable." But why?
>
> Pira pointed out that the US's approach brings enormous economic
> benefits. The US and EU are comparable in size and population; but
> while the EU spent €9.5bn (£6.51bn) on gathering public sector data,
> and collected €68bn selling and licensing it, the US spent €19bn -
> twice as much - and realised €750bn - over 10 times more. Weiss
> pointed out: "Governments realise two kinds of financial gain when
> they drop charges: higher indirect tax revenue from higher sales of
> the products that incorporate the ... information; and higher income
> tax revenue and lower social welfare payments from net gains in
> employment."
>
> Infinite applications
>
> The applications for public sector data are infinite, but here are two
> real-world ones that affect ordinary businesses. Many of Britain's
> best rock-climbing venues are on sea cliffs, and hence affected by the
> tides. For climbers planning a trip - and surely spending money in
> local shops - it helps to know if the tides will be favourable. But
> websites that try to offer British tide data have been told by the UK
> Hydrographic Office they must pay for it - a cost most are unwilling
> to endure. So sites have no tides, climbers make the safer choice, and
> shops miss out.
>
> Take traffic data. The Highways Agency has set up an exclusive deal
> with TrafficTV to send video details from motorway cameras to your
> mobile. But if the data were free, any company could offer
> well-organised maps and drive down the price while increasing the
> numbers who would have better access to traffic data. That would
> reduce congestion and pollution.
>
> How did we get into this mess? The problem stems from the days of the
> Conservative government, which created "trading funds". These
> government-owned bodies - including OS, Companies House, the Driving
> Standards Agency, HM Land Registry, the Patent Office and the Royal
> Mint - have to sell to other parts of the government and private
> customers. Every year they have a "revenue target" that must be paid
> to the Treasury; but the only way to earn those revenues is through
> commercial activity - creating the ludicrous spectacle of
> government-approved monopolies making the market buy data at their
> rates. A growing number of experts say the loss to the economy
> outweighs "gains" through lowered taxes. "The amount of money made is
> trivial," says Keith Dugmore of the Demographics User Group, which
> represents commercial users of government information.
>
> But the stifling of competition is most serious. Sometimes, the
> conflicts are surreal. The supply of postal addresses is an example.
> Three publicly owned agencies - the Post Office, OS and a consortium
> of local authorities - run lists of addresses. Because they are forced
> to put a "value" on the lists, they have been unable to agree a way of
> creating a single database. The cost of licensing addresses was one
> reason why returns from the 2001 census were so poor from several
> fast-changing metropolitan areas, such as Manchester.
>
> A local authority such as Swindon has to pay OS £38,000 a year to use
> its addresses and geographical data. It also has to pay the Royal Mail
> £3,000 for every website that includes the facility for people to look
> up their postcodes. Yet it was local authorities, which have a
> statutory duty to collect street addresses, that collected much of
> this data.
>
> Intense conflict
>
> Kristin Woodland, who chairs the local authorities' street gazetteer
> group, says: "The taxpayer pays for us to create the data, then has to
> pay us to use the data."
>
> The conflict between a trading fund's traditional role and its new
> commercial activities is most intense in mapping. "Technology has left
> the licensing model far behind," says Dugmore.
>
> Particular controversy surrounds the role of OS, founded to prepare
> the British army for a French invasion and now a household name that
> sees itself at the forefront of geographical information systems.
>
> OS has a monopoly on centuries' worth of data thanks to taxpayer
> funding. Without that head start, it would be in the same position as
> any other startup today: facing the choice of creating a brand-new
> dataset, or finding someone who had collected it and licensing it from
> them.
>
> Part of the problem is that government - and the Treasury - is blinded
> by its internal commercial targets, and unable to see the damage its
> policies do. Thus within government, OS is widely seen as an example
> of a venerable public body transforming itself into a dynamic
> commercial enterprise. Last year it had a surplus of £9.2m, though
> only part was returned to the Treasury. But the bulk of its revenues
> come from the public sector, partly through a payment called the
> National Interest Mapping Services (Nims), worth £13m a year, and
> partly from deals with government bodies.
>
> The latest of the deals is under negotiation. Yesterday, the
> government was due to receive bids for a £20m contract to provide
> geographical information to Whitehall bodies. In theory, it is an open
> competition. In practice, the largest part of the contract, for "large
> scale data", is unlikely to go to anyone else: the contract specifies
> that the winner must have large-scale maps in place within six weeks
> of the start of a five-year deal.
>
> The OS told the Guardian: "We are bidding for pan-government agreement
> and we operate in fair competition under heavy scrutiny. The only
> monopoly we have is in the mapping of uneconomic areas." In 2004-05,
> 47.5% of its £100.4m trading revenue came from government (which
> allows it to claim that it is not government funded). It says Nims is
> not a subsidy: "This is a contribution to the cost of mapping
> uneconomic areas."
>
> Profit centres
>
> That makes sense - if you consider the geography of the United Kingdom
> to be a profit centre. But if you view it instead as belonging to its
> taxpayers, and meriting rigorous mapping for their benefit, there are
> no "uneconomic areas" - only places that people haven't started to use
> yet.
>
> Happily, the practice of state-owned monopolies competing in markets
> dependent on their information is under attack from several quarters.
> A new trade association, Locus, is calling for the government to
> enforce a level playing field in the market in public sector
> information.
>
> And some of the most copyright-mad government organisations have
> relented. Hansard, the record of proceedings in parliament, was once a
> paper product that had to be bought from Her Majesty's Stationery
> Office. Now it is online, for free, so that sites like
> theyworkforyou.com can generate useful data about individual MPs'
> voting and attendance.
>
> The Office of Fair Trading is preparing a report on public sector
> information, due this summer, which will "look at whether or not the
> way in which public sector information holders (PSIH) supply
> information works well for businesses. It will examine whether PSIHs
> have an unfair advantage selling on information in competition with
> companies who are reliant on the PSIH for that raw data in the first
> place."
>
> Though it may already be shooting for an open goal, we urge the OFT to
> compare the UK with the US; read Peter Weiss's paper; and then,
> finally, to free our data.
>
> _______________________________________________
> CivicAccess-discuss mailing list
> [hidden email]
> http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca
>