New site: openparliament.ca

[Michael Mulley]

Hello civic accessors,

I've put up a new Parliamentary data site you may be interested in:
http://openparliament.ca/

At the moment, it's largely an interface and search engine on top of
scraped Hansard data, with some aggregation of politician information.
I'm also really happy to be making available parsed XML and JSON
versions of House debates dating back to 1994.

I just added an advocacy blurb for open data to the about page; if the
folks on the list have any suggested revisions to that, I'd love to
hear them.

michael

[Glen Newton]

Very nice work. A great start!

Checked-out the API.
Some suggestions:
- I would include an RDF version, so that this resource can play in
the Linked Data universe[1]
- The Hansard API contains extensive redundant information. Example:
Every time a politician speaks, we get:
 <politician>
<name>Peter Milliken</name>
<url>/politicians/peter-milliken/</url>
<member_id>3187</member_id>
<party>Liberal</party>
<riding>Kingston and the Islands (Ontario)</riding>
<id>203</id>
</politician>

This can bloat the files coming back.

I think it would be preferable to have an API also for politicians,
and in this example replace the above XML with:
 <politician id=3187 />

The Politician API would look something like:
http://openparliament.ca/api/politician/3187/?format=xml

And would return XML similar to the above original.

Also, what are you using for the search engine? SOLR?

thanks,
Glen
http://zzzoot.blogspot.com/

[1]http://linkeddata.org/


On 12 April 2010 11:48, Michael Mulley <[hidden email]> wrote:

> Hello civic accessors,
>
> I've put up a new Parliamentary data site you may be interested in:
> http://openparliament.ca/
>
> At the moment, it's largely an interface and search engine on top of
> scraped Hansard data, with some aggregation of politician information.
> I'm also really happy to be making available parsed XML and JSON
> versions of House debates dating back to 1994.
>
> I just added an advocacy blurb for open data to the about page; if the
> folks on the list have any suggested revisions to that, I'd love to
> hear them.
>
> michael
> _______________________________________________
> CivicAccess-discuss mailing list
> [hidden email]
> http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>

--

-

[Michael Lenczner]

It looks great!  :)

I've been continuing work on YAHP with a few folks on this list. We'll
share it soon.  Michael and a few others saw it recently at a small
hack-day we had a couple of weeks ago.

I'm looking forward to chatting over these projects with whoever is
around at the Ottawa Open Data day. (http://opendataottawa.ca/)

On Mon, Apr 12, 2010 at 11:48 AM, Michael Mulley
<[hidden email]> wrote:

> Hello civic accessors,
>
> I've put up a new Parliamentary data site you may be interested in:
> http://openparliament.ca/
>
> At the moment, it's largely an interface and search engine on top of
> scraped Hansard data, with some aggregation of politician information.
> I'm also really happy to be making available parsed XML and JSON
> versions of House debates dating back to 1994.
>
> I just added an advocacy blurb for open data to the about page; if the
> folks on the list have any suggested revisions to that, I'd love to
> hear them.
>
> michael
> _______________________________________________
> CivicAccess-discuss mailing list
> [hidden email]
> http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>

--
http://michaellenczner.ca
http://twitter.com/mlenc

[Michael Lenczner]

And great blurb, Michael!  Wow.  I'm going to copy it here so that
everyone can check it out.  There are great hyperlinks, so you should
read it actually on the OpenParliament site.

http://openparliament.ca/about/

"What Open means
Even if the battle's far from won, the case for transparency in
government is clear: of course we should know how our representatives
are representing us, of course we should be able to see what's being
done with our tax dollars. But, as information and data become
increasingly synonymous, making information available isn't enough. To
be useful, it has to be usable. It has to be freely available, in a
flexible digital format. It has to be open.

Take the House of Commons transcripts that make up the bulk of this
site. Parliament has transcripts since 1994 online, which is great.
But they're available only as pre-formatted Web pages, which means
that to get the data I had to construct a wobbly tower of rules—if
it's a 14-pixel font, it's probably a person's name, as long as it's
not within a table and it doesn't contain the words The, Some, One, or
An, or Assistant—that took many days and more frustration to get
right. And if Parliament starts to use a different font size, this
site stops working.

Meanwhile, the House is kind enough to make vote information available
in a simple, open format. I used that data on this site, where it's
presented in a way that I find clearer than on Parliament's site. It
took me an afternoon.

But this isn't about making things convenient for me. It's about the
innovation that data can unlock—and the obstacles closed data puts in
democracy's way. In 2003, the federal government declared that travel
and hospitality expenses would be public. But then each department
made its reports available in different places, and in different
clunky, error-ridden, non-open formats. The information is unusable,
unless you're both patient and valiant.

Opening up data ain't all that hard. Once I had the House transcripts
processed—that is, back into the kind of structured form in which it
lives, hidden from the public, in Parliament's computers—opening the
data took a couple of hours. But, despite the benefits, too often
government just doesn't care. When transit agencies in Halifax and
Toronto wouldn't make their schedule information open, brilliant
programmers reverse-engineered the schedule data and built immensely
useful trip-planning sites. The transit agencies' reaction? Continued
indifference.

You can do great stuff with prosaic municipal data, and some cities,
from Nanaimo to Toronto, are passing open data resolutions and
starting to share information. Things are looking up. Communities are
forming. But, unlike other countries, Canada doesn't yet have a
federal open data plan or culture. We can do better."

On Mon, Apr 12, 2010 at 1:59 PM, Michael Lenczner <[hidden email]> wrote:

> It looks great!  :)
>
> I've been continuing work on YAHP with a few folks on this list. We'll
> share it soon.  Michael and a few others saw it recently at a small
> hack-day we had a couple of weeks ago.
>
> I'm looking forward to chatting over these projects with whoever is
> around at the Ottawa Open Data day. (http://opendataottawa.ca/)
>
> On Mon, Apr 12, 2010 at 11:48 AM, Michael Mulley
> <[hidden email]> wrote:
>> Hello civic accessors,
>>
>> I've put up a new Parliamentary data site you may be interested in:
>> http://openparliament.ca/
>>
>> At the moment, it's largely an interface and search engine on top of
>> scraped Hansard data, with some aggregation of politician information.
>> I'm also really happy to be making available parsed XML and JSON
>> versions of House debates dating back to 1994.
>>
>> I just added an advocacy blurb for open data to the about page; if the
>> folks on the list have any suggested revisions to that, I'd love to
>> hear them.
>>
>> michael
>> _______________________________________________
>> CivicAccess-discuss mailing list
>> [hidden email]
>> http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>>
>
>
>
> --
> http://michaellenczner.ca
> http://twitter.com/mlenc
>

--
http://michaellenczner.ca
http://twitter.com/mlenc

[Michael Mulley]

Hi Glen,

Thanks!

On the tech stuff:

- Yeah, RDF is on my list of things-I'd-like-to-learn-about anyway,
and is definitely on my wishlist for the site. I've never worked with
it before, though, so this'll probably wait until I have the time to
sit down with a few books.

- API is a bit of a misnomer for what's up there now, really. It's
more of a dump of my parsed data, intended to be easy for a programmer
to run a one-time import on, so being concise wasn't really a goal. As
demand warrants, I'd definitely like to create an actual API designed
for live usage.

- Yeah, the search is Solr. Which means that there's the backend
capability to add more advanced search features than what's up at the
moment.

michael

On Mon, Apr 12, 2010 at 1:23 PM, Glen Newton <[hidden email]> wrote:

> Very nice work. A great start!
>
> Checked-out the API.
> Some suggestions:
> - I would include an RDF version, so that this resource can play in
> the Linked Data universe[1]
> - The Hansard API contains extensive redundant information. Example:
> Every time a politician speaks, we get:
>  <politician>
> <name>Peter Milliken</name>
> <url>/politicians/peter-milliken/</url>
> <member_id>3187</member_id>
> <party>Liberal</party>
> <riding>Kingston and the Islands (Ontario)</riding>
> <id>203</id>
> </politician>
>
> This can bloat the files coming back.
>
> I think it would be preferable to have an API also for politicians,
> and in this example replace the above XML with:
>  <politician id=3187 />
>
> The Politician API would look something like:
> http://openparliament.ca/api/politician/3187/?format=xml
>
> And would return XML similar to the above original.
>
> Also, what are you using for the search engine? SOLR?
>
> thanks,
> Glen
> http://zzzoot.blogspot.com/
>
> [1]http://linkeddata.org/
>
>
> On 12 April 2010 11:48, Michael Mulley <[hidden email]> wrote:
>> Hello civic accessors,
>>
>> I've put up a new Parliamentary data site you may be interested in:
>> http://openparliament.ca/
>>
>> At the moment, it's largely an interface and search engine on top of
>> scraped Hansard data, with some aggregation of politician information.
>> I'm also really happy to be making available parsed XML and JSON
>> versions of House debates dating back to 1994.
>>
>> I just added an advocacy blurb for open data to the about page; if the
>> folks on the list have any suggested revisions to that, I'd love to
>> hear them.
>>
>> michael
>> _______________________________________________
>> CivicAccess-discuss mailing list
>> [hidden email]
>> http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>>
>
>
>
> --
>
> -
> _______________________________________________
> CivicAccess-discuss mailing list
> [hidden email]
> http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
>

[Ted Hildebrandt]

Nice article in the Globe & Mail on openparliment.ca and howdtheyvote.ca. Bring it on!

http://www.theglobeandmail.com/news/politics/if-you-wont-tell-us-about-our-mps-well-do-it-for-you/article1532700/



---------------------------------------
Ted Hildebrandt
Director of Social Planning
Community Development Halton
860 Harrington Court
Burlington, Ontario  L7N 3N4 Canada
Phone: [phone]
Fax: [phone]
Email: [hidden email]
Web: www.cdhalton.ca
           www.volunteerhalton.ca

Building Community Together