Posted by
Russell McOrmond-2 on
Mar 25, 2007; 10:42pm
URL: http://civicaccess.416.s1.nabble.com/MPs-by-postal-code-tp959p974.html
Hugh McGuire wrote:
>> I'm only
>> caching results from the postal code lookups, and not attempting to
>> build a complete database, so I believe I'm in the clear for that too.
> OK so the project to build an open, free database of ridings v postal
> codes is still desirable?
Yes, this is still needed. Screen scraping is a very ugly kludge
that doesn't solve the underlying technical, legal or political problems.
Elections Canada deliberately tries to break screen scraping, and has
randomly changes the method they use over recent years to kill screen
scraping tools (Drupal's Lobby module, the ECTOOLS tool that I used in
the past, etc).
http://sourceforge.net/projects/campaigntoolz/ ECTools was a PHP system which used XML-RPC to split a caching server
that would screen scrape from a small client which would do a database
lookup. The idea was to have many sites using the client, and one site
running the caching server. This never really worked well as the screen
scraping kept breaking and it was very hard to keep it up-to-date.
http://campaigntoolz.cvs.sourceforge.net/campaigntoolz/ectools/ The parl.gc.ca site won't work during the election, or at least it
was shut down in the past. This would work if there was a secondary
database that converted the incumbent MP link into an Electoral District
that could then be used as the index against the current candidates
database. Expect parl.gc.ca to follow elections.ca to shutting things
down if screen scraping becomes common.
We need this information to be released directly with a clear open
license so that it can be shared and imported without the problems that
screen scraping has (IE: may work this moment, may be dead a minute from
now).
While IANAL, I believe this screen scraping is a clear copyright
infringement, but one where the copyright holder is quite unlikely to
sue for infringement. The bad politics of Elections Canada or the
Library of Parliament sueing someone for screen scraping this data could
even be a win for us as the data is then made public legally.
One-time screen scraping like the collection of the contact and other
information for MPs is different in that we can scrape, verify the data,
and publish the results as has been done. We don't need to rely on the
parl.gc.ca site letting us in tomorrow as we already have the relevant
information today.
Note: During elections Elections Canada releases a database of all
candidates, which is what makes sites that list candidates so easy. It
is unfortunate that parl.gc.ca doesn't already do this for sitting MPs
(more reliable than screen scraping), and that the postal code database
needs to be released.
Some interesting stuff from Elections Canada to be aware of, especially
if we are heading into an election (Possibly over the Clean Air Act
since the budget won't be an issue).
Final List of Confirmed Candidates – 39th General Election (This is
live updated during the election, and can be imported directly into a
database)
http://www.elections.ca/content.asp?section=pas&document=index&dir=39ge/loc&lang=e&textonly=false Here is a tool I wrote to allow people to browse this database (with
additions of websites/email contact that was done by our community
during the election)
http://www.digital-copyright.ca/election2006/candidatesOfficial Voting Results of the 39th General Election – Poll-by-Poll
Results – Raw Data
http://www.elections.ca/scripts/resval/ovr_39ge.asp?prov=&lang=e...
--
Russell McOrmond, Internet Consultant: <
http://www.flora.ca/>
Please help us tell the Canadian Parliament to protect our property
rights as owners of Information Technology. Sign the petition!
http://www.digital-copyright.ca/petition/ict/ "The government, lobbied by legacy copyright holders and hardware
manufacturers, can pry my camcorder, computer, home theatre, or
portable media player from my cold dead hands!"