Zip codes and Electoral districts

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Zip codes and Electoral districts

Stéphane Zagar
Hey

In fact, technically it's not complicated to get a zip code mapping with latitude and longitude :

H3T1A1;45.5113;-73.6162
H3T1B1;45.5083;-73.6183
H3T1C1;45.5016;-73.6245
H3T1E1;45.4974;-73.628
H3T1G1;45.51;-73.614
H3T1H1;45.5061;-73.6176
H3T1J1;45.4945;-73.6282
H3T1K1;45.5002;-73.6212
H3T1L1;45.4962;-73.6253
H3T1M1;45.4971;-73.6225
H3T1N1;45.501;-73.6178
H3T1P1;45.4942;-73.6229
H3T1R1;45.5105;- 73.6154
H3T1S1;45.5078;-73.6178
H3T1T1;45.5052;-73.6198
H3T1V1;45.5005;-73.6192
H3T1W1;45.4987;-73.6208
H3T1X1;45.4971;-73.6212
H3T1Y1;45.4959;-73.6206
H3T1Z1;45.4977;-73.6257

And I think it's not that much complicated to do the same reverse engineering for electoral district.

The problem is that mathematically, there are about 17 millions of valid combination of zip code. It take about 1 sec for my script to retrieve the zipcode/lat-long so if we do a basic computation it gives us about... 100 days to retrieve everything (and probably a lot of bandwidth). Even if we set the last number to 1 (what I did in my previous test), it's still about 2 millions valid values. And I'm not sure if the website I use to retrieve this will allow this (with 1 request/sec, I can't imagine they won't see that something is happening).

By the way, I have a question for geostuff people (Tracey ?) : On the geogratis website, there are some files concerning the electoral districts ( http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169), but I don't know anything concerning the format (Arc Export or shape file). Is there a way to do something with that ? For example do a process what-ever to find out in which electoral district is a point (long-lat) ?

Steph
Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Daniel Haran
Hi Stéphane,

There are a few problems with that approach. First of all, the
lat/long mapping databases tend to be copyrighted themselves. Besides,
those are the centroids - the center-most point of a polygon.

One of the main problems Russell mentioned (and explained further at
http://www.digital-copyright.ca/node/1607 ) is that the mapping is not
one to one. A postal code region can overlap several electoral
districts.

The drupal lobby module he mentioned has a good regular expression for
validating postal codes:
/([ABCEGHJKLMNPRSTVXY]\d[A-Z]\s?\d[A-Z]\d)/i

For non-techies, that first segment ABC...Y are all the allowable
first letters in a postal code. That reduces the number of
possibilities to 1,216,800 from 17,576,000. At a second a piece,
that's a bit over 14 days from 100 :)

cheers,

Daniel.

On 2/8/07, Stéphane Zagar <[hidden email]> wrote:

> Hey
>
> In fact, technically it's not complicated to get a zip code mapping with
> latitude and longitude :
>
> H3T1A1;45.5113;-73.6162
> H3T1B1;45.5083;-73.6183
> H3T1C1;45.5016;-73.6245
> H3T1E1;45.4974;-73.628
> H3T1G1;45.51;-73.614
> H3T1H1;45.5061;-73.6176
> H3T1J1;45.4945;-73.6282
> H3T1K1;45.5002;-73.6212
> H3T1L1;45.4962;-73.6253
> H3T1M1;45.4971;-73.6225
> H3T1N1;45.501;-73.6178
> H3T1P1;45.4942;-73.6229
> H3T1R1;45.5105;- 73.6154
> H3T1S1;45.5078;-73.6178
> H3T1T1;45.5052;-73.6198
> H3T1V1;45.5005;-73.6192
> H3T1W1;45.4987;-73.6208
> H3T1X1;45.4971;-73.6212
> H3T1Y1;45.4959;-73.6206
> H3T1Z1;45.4977;-73.6257
>
> And I think it's not that much complicated to do the same reverse
> engineering for electoral district.
>
> The problem is that mathematically, there are about 17 millions of valid
> combination of zip code. It take about 1 sec for my script to retrieve the
> zipcode/lat-long so if we do a basic computation it gives us about... 100
> days to retrieve everything (and probably a lot of bandwidth). Even if we
> set the last number to 1 (what I did in my previous test), it's still about
> 2 millions valid values. And I'm not sure if the website I use to retrieve
> this will allow this (with 1 request/sec, I can't imagine they won't see
> that something is happening).
>
> By the way, I have a question for geostuff people (Tracey ?) : On the
> geogratis website, there are some files concerning the electoral districts (
> http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169),
> but I don't know anything concerning the format (Arc Export or shape file).
> Is there a way to do something with that ? For example do a process
> what-ever to find out in which electoral district is a point (long-lat) ?
>
> Steph
>
> _______________________________________________
> CivicAccess-discuss mailing list
> [hidden email]
> http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Stéphane Zagar
Daniel :

I don't see how this can be copyrighted. Someone (you Daniel ?) told that facts can't be copyrighted. I just can't imagine how this could be copyrighted ! Damned I could call every and each Canadien citizen and ask him his zipcode and his electoral district, then can't use this ?

Concerning the other points you raise :
- It's amazing that postal codes can overlap several districts... but I can't see how to solve this. Even the Election Canada website uses the postal code to find your district.
- I don't see how you get only 1 millions possibilites with this regex. You bring the first character from 26 letters to 19. The result is 19*10*26*10*26*10 = 12 millions


Franck :

I understand you point. However I still don't understand why those data are not freely accessible. If they are not available I'll pick them... if I can ;)

Steph

Daniel Haran wrote:
Hi Stéphane,

There are a few problems with that approach. First of all, the
lat/long mapping databases tend to be copyrighted themselves. Besides,
those are the centroids - the center-most point of a polygon.

One of the main problems Russell mentioned (and explained further at
http://www.digital-copyright.ca/node/1607 ) is that the mapping is not
one to one. A postal code region can overlap several electoral
districts.

The drupal lobby module he mentioned has a good regular expression for
validating postal codes:
/([ABCEGHJKLMNPRSTVXY]\d[A-Z]\s?\d[A-Z]\d)/i

For non-techies, that first segment ABC...Y are all the allowable
first letters in a postal code. That reduces the number of
possibilities to 1,216,800 from 17,576,000. At a second a piece,
that's a bit over 14 days from 100 :)

cheers,

Daniel.

On 2/8/07, Stéphane Zagar [hidden email] wrote:
  
Hey

In fact, technically it's not complicated to get a zip code mapping with
latitude and longitude :

H3T1A1;45.5113;-73.6162
H3T1B1;45.5083;-73.6183
H3T1C1;45.5016;-73.6245
H3T1E1;45.4974;-73.628
H3T1G1;45.51;-73.614
H3T1H1;45.5061;-73.6176
H3T1J1;45.4945;-73.6282
H3T1K1;45.5002;-73.6212
H3T1L1;45.4962;-73.6253
H3T1M1;45.4971;-73.6225
H3T1N1;45.501;-73.6178
H3T1P1;45.4942;-73.6229
H3T1R1;45.5105;- 73.6154
H3T1S1;45.5078;-73.6178
H3T1T1;45.5052;-73.6198
H3T1V1;45.5005;-73.6192
H3T1W1;45.4987;-73.6208
H3T1X1;45.4971;-73.6212
H3T1Y1;45.4959;-73.6206
H3T1Z1;45.4977;-73.6257

And I think it's not that much complicated to do the same reverse
engineering for electoral district.

The problem is that mathematically, there are about 17 millions of valid
combination of zip code. It take about 1 sec for my script to retrieve the
zipcode/lat-long so if we do a basic computation it gives us about... 100
days to retrieve everything (and probably a lot of bandwidth). Even if we
set the last number to 1 (what I did in my previous test), it's still about
2 millions valid values. And I'm not sure if the website I use to retrieve
this will allow this (with 1 request/sec, I can't imagine they won't see
that something is happening).

By the way, I have a question for geostuff people (Tracey ?) : On the
geogratis website, there are some files concerning the electoral districts (
http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169),
but I don't know anything concerning the format (Arc Export or shape file).
Is there a way to do something with that ? For example do a process
what-ever to find out in which electoral district is a point (long-lat) ?

Steph

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca


    

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca

  

Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Daniel Haran
On 2/8/07, Stéphane Zagar <[hidden email]> wrote:
>
>  Daniel :
>
>  I don't see how this can be copyrighted. Someone (you Daniel ?) told that
> facts can't be copyrighted. I just can't imagine how this could be
> copyrighted ! Damned I could call every and each Canadien citizen and ask
> him his zipcode and his electoral district, then can't use this ?

You sure could. AFAIK though, the centroid databases I've seen are all
copyrighted and prohibit derivative database creation. Stupid, I know.

>  Concerning the other points you raise :
>  - It's amazing that postal codes can overlap several districts... but I
> can't see how to solve this. Even the Election Canada website uses the
> postal code to find your district.

Hmmm...
http://www.parl.gc.ca/information/about/people/house/PostalCode.asp?Language=E&txtPostalCode=H1T4C6

elections.ca seems to be giving people the wrong information.... hey,
what do you know, maybe duplication of work is even misleading
Canadians.

>  - I don't see how you get only 1 millions possibilites with this regex. You
> bring the first character from 26 letters to 19. The result is
> 19*10*26*10*26*10 = 12 millions

Ack... I need more caffeine while doing math, and I need to stop using
string sizes to count items. :(

cheers,

Daniel.


Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Stéphane Zagar
Daniel Haran wrote:


Hmmm...
http://www.parl.gc.ca/information/about/people/house/PostalCode.asp?Language=E&txtPostalCode=H1T4C6

elections.ca seems to be giving people the wrong information.... hey,
what do you know, maybe duplication of work is even misleading
Canadians.
  
That's incredible, 5 districts at the same time ! If you seach this postal code in Google and Election Canada, you'll see that the corresponding point is just between Hochelaga and Saint-Leonard districts. But it's really far from the others. A complete postal code in a city like Montreal can't have a size greater than hunder meters (I know many people on my streets, even those that are close don't have the same postal code.  I can't see how it's possible.
Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Tracey P. Lauriault-2
In reply to this post by Stéphane Zagar
Stéphane Zagar wrote:
> Daniel :
>
> I don't see how this can be copyrighted. Someone (you Daniel ?) told
> that facts can't be copyrighted. I just can't imagine how this could
> be copyrighted ! Damned I could call every and each Canadien citizen
> and ask him his zipcode and his electoral district, then can't use this ?
it is a unique arrangement of facts.  Most databases are full of facts,
rivers, households, telephone numbers.  It is not the fact but the
unique arrangement of these facts that make them copyrightable.

>
> Concerning the other points you raise :
> - It's amazing that postal codes can overlap several districts... but
> I can't see how to solve this. Even the Election Canada website uses
> the postal code to find your district.
> - I don't see how you get only 1 millions possibilites with this
> regex. You bring the first character from 26 letters to 19. The result
> is 19*10*26*10*26*10 = 12 millions
>
>
> Franck :
>
> I understand you point. However I still don't understand why those
> data are not freely accessible. If they are not available I'll pick
> them... if I can ;)
>
> Steph
>
> Daniel Haran wrote:
>> Hi Stéphane,
>>
>> There are a few problems with that approach. First of all, the
>> lat/long mapping databases tend to be copyrighted themselves. Besides,
>> those are the centroids - the center-most point of a polygon.
>>
>> One of the main problems Russell mentioned (and explained further at
>> http://www.digital-copyright.ca/node/1607 ) is that the mapping is not
>> one to one. A postal code region can overlap several electoral
>> districts.
>>
>> The drupal lobby module he mentioned has a good regular expression for
>> validating postal codes:
>> /([ABCEGHJKLMNPRSTVXY]\d[A-Z]\s?\d[A-Z]\d)/i
>>
>> For non-techies, that first segment ABC...Y are all the allowable
>> first letters in a postal code. That reduces the number of
>> possibilities to 1,216,800 from 17,576,000. At a second a piece,
>> that's a bit over 14 days from 100 :)
>>
>> cheers,
>>
>> Daniel.
>>
>> On 2/8/07, Stéphane Zagar <[hidden email]> wrote:
>>  
>>> Hey
>>>
>>> In fact, technically it's not complicated to get a zip code mapping with
>>> latitude and longitude :
>>>
>>> H3T1A1;45.5113;-73.6162
>>> H3T1B1;45.5083;-73.6183
>>> H3T1C1;45.5016;-73.6245
>>> H3T1E1;45.4974;-73.628
>>> H3T1G1;45.51;-73.614
>>> H3T1H1;45.5061;-73.6176
>>> H3T1J1;45.4945;-73.6282
>>> H3T1K1;45.5002;-73.6212
>>> H3T1L1;45.4962;-73.6253
>>> H3T1M1;45.4971;-73.6225
>>> H3T1N1;45.501;-73.6178
>>> H3T1P1;45.4942;-73.6229
>>> H3T1R1;45.5105;- 73.6154
>>> H3T1S1;45.5078;-73.6178
>>> H3T1T1;45.5052;-73.6198
>>> H3T1V1;45.5005;-73.6192
>>> H3T1W1;45.4987;-73.6208
>>> H3T1X1;45.4971;-73.6212
>>> H3T1Y1;45.4959;-73.6206
>>> H3T1Z1;45.4977;-73.6257
>>>
>>> And I think it's not that much complicated to do the same reverse
>>> engineering for electoral district.
>>>
>>> The problem is that mathematically, there are about 17 millions of valid
>>> combination of zip code. It take about 1 sec for my script to retrieve the
>>> zipcode/lat-long so if we do a basic computation it gives us about... 100
>>> days to retrieve everything (and probably a lot of bandwidth). Even if we
>>> set the last number to 1 (what I did in my previous test), it's still about
>>> 2 millions valid values. And I'm not sure if the website I use to retrieve
>>> this will allow this (with 1 request/sec, I can't imagine they won't see
>>> that something is happening).
>>>
>>> By the way, I have a question for geostuff people (Tracey ?) : On the
>>> geogratis website, there are some files concerning the electoral districts (
>>> http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169),
>>> but I don't know anything concerning the format (Arc Export or shape file).
>>> Is there a way to do something with that ? For example do a process
>>> what-ever to find out in which electoral district is a point (long-lat) ?
>>>
>>> Steph
>>>
>>> _______________________________________________
>>> CivicAccess-discuss mailing list
>>> [hidden email]
>>> http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca
>>>
>>>
>>>    
>>
>> _______________________________________________
>> CivicAccess-discuss mailing list
>> [hidden email]
>> http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca
>>
>>  
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> CivicAccess-discuss mailing list
> [hidden email]
> http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca
>  



Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Frank Warmerdam
In reply to this post by Stéphane Zagar
Stéphane Zagar wrote:
> Hey
>
> In fact, technically it's not complicated to get a zip code mapping with
> latitude and longitude :
>
> H3T1A1;45.5113;-73.6162
...

> H3T1Z1;45.4977;-73.6257
>
> And I think it's not that much complicated to do the same reverse
> engineering for electoral district.
>
> The problem is that mathematically, there are about 17 millions of valid
> combination of zip code. It take about 1 sec for my script to retrieve
> the zipcode/lat-long so if we do a basic computation it gives us
> about... 100 days to retrieve everything (and probably a lot of
> bandwidth). Even if we set the last number to 1 (what I did in my
> previous test), it's still about 2 millions valid values. And I'm not
> sure if the website I use to retrieve this will allow this (with 1
> request/sec, I can't imagine they won't see that something is happening).

Stéphane,

I'm afraid I haven't been following this thread very closely despite the
fact that it is close to my heart.  But I'd assume the terms of use for
the site do not allow you to extract their full database this way and
redistribute it.  Is that not the case?

> By the way, I have a question for geostuff people (Tracey ?) : On the
> geogratis website, there are some files concerning the electoral
> districts (
> http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169), but
> I don't know anything concerning the format (Arc Export or shape file).
> Is there a way to do something with that ? For example do a process
> what-ever to find out in which electoral district is a point (long-lat) ?

I can definitely help with this sort of thing if you want to do it.
Geo-data-wrangling is my thing.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org



Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Stéphane Zagar
Hi Frank,

As someone else on the list said, zip codes and electoral districts are a fact, there's no way for them to know how I build my mapping.

In fact I would be happy to send this mapping to anybody who want to sue me with that to see what would be the result. But for the moment, I'm not sure if I would be able to get everything since the website I use might declare my IP address as fraudulent after only 1000 zip retrieved :p

Steph

Frank Warmerdam wrote:
Stéphane Zagar wrote:
  
Hey

In fact, technically it's not complicated to get a zip code mapping with 
latitude and longitude :

H3T1A1;45.5113;-73.6162
    
...
  
H3T1Z1;45.4977;-73.6257

And I think it's not that much complicated to do the same reverse 
engineering for electoral district.

The problem is that mathematically, there are about 17 millions of valid 
combination of zip code. It take about 1 sec for my script to retrieve 
the zipcode/lat-long so if we do a basic computation it gives us 
about... 100 days to retrieve everything (and probably a lot of 
bandwidth). Even if we set the last number to 1 (what I did in my 
previous test), it's still about 2 millions valid values. And I'm not 
sure if the website I use to retrieve this will allow this (with 1 
request/sec, I can't imagine they won't see that something is happening).
    

Stéphane,

I'm afraid I haven't been following this thread very closely despite the
fact that it is close to my heart.  But I'd assume the terms of use for
the site do not allow you to extract their full database this way and
redistribute it.  Is that not the case?

  
By the way, I have a question for geostuff people (Tracey ?) : On the 
geogratis website, there are some files concerning the electoral 
districts ( 
http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169), but 
I don't know anything concerning the format (Arc Export or shape file). 
Is there a way to do something with that ? For example do a process 
what-ever to find out in which electoral district is a point (long-lat) ?
    

I can definitely help with this sort of thing if you want to do it.
Geo-data-wrangling is my thing.

Best regards,
  

Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Tracey P. Lauriault-2
In reply to this post by Frank Warmerdam
the lat long you get is a point, while the 6 digit postal code is a
polygon!  So for the data to be useful you also need the coordinates of
intersecting lines.


Frank Warmerdam wrote:

> Stéphane Zagar wrote:
>  
>> Hey
>>
>> In fact, technically it's not complicated to get a zip code mapping with
>> latitude and longitude :
>>
>> H3T1A1;45.5113;-73.6162
>>    
> ...
>  
>> H3T1Z1;45.4977;-73.6257
>>
>> And I think it's not that much complicated to do the same reverse
>> engineering for electoral district.
>>
>> The problem is that mathematically, there are about 17 millions of valid
>> combination of zip code. It take about 1 sec for my script to retrieve
>> the zipcode/lat-long so if we do a basic computation it gives us
>> about... 100 days to retrieve everything (and probably a lot of
>> bandwidth). Even if we set the last number to 1 (what I did in my
>> previous test), it's still about 2 millions valid values. And I'm not
>> sure if the website I use to retrieve this will allow this (with 1
>> request/sec, I can't imagine they won't see that something is happening).
>>    
>
> Stéphane,
>
> I'm afraid I haven't been following this thread very closely despite the
> fact that it is close to my heart.  But I'd assume the terms of use for
> the site do not allow you to extract their full database this way and
> redistribute it.  Is that not the case?
>
>  
>> By the way, I have a question for geostuff people (Tracey ?) : On the
>> geogratis website, there are some files concerning the electoral
>> districts (
>> http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169), but
>> I don't know anything concerning the format (Arc Export or shape file).
>> Is there a way to do something with that ? For example do a process
>> what-ever to find out in which electoral district is a point (long-lat) ?
>>    
>
> I can definitely help with this sort of thing if you want to do it.
> Geo-data-wrangling is my thing.
>
> Best regards,
>  



Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Stéphane Zagar
I know but I'm not a magician :)

I don't see any way to get that.
My question is to know whether it's useful to be that precise. Maybe my view is biaised because I take my examples in Montreal, but a 6 digit postal code is a very small polygon. If I just want to match a postal code (a point) within a district (an area), it should be possible. No ? Is it an ugly way to do it ?

Steph

Tracey P. Lauriault wrote:
the lat long you get is a point, while the 6 digit postal code is a 
polygon!  So for the data to be useful you also need the coordinates of 
intersecting lines.


Frank Warmerdam wrote:
  
Stéphane Zagar wrote:
  
    
Hey

In fact, technically it's not complicated to get a zip code mapping with 
latitude and longitude :

H3T1A1;45.5113;-73.6162
    
      
...
  
    
H3T1Z1;45.4977;-73.6257

And I think it's not that much complicated to do the same reverse 
engineering for electoral district.

The problem is that mathematically, there are about 17 millions of valid 
combination of zip code. It take about 1 sec for my script to retrieve 
the zipcode/lat-long so if we do a basic computation it gives us 
about... 100 days to retrieve everything (and probably a lot of 
bandwidth). Even if we set the last number to 1 (what I did in my 
previous test), it's still about 2 millions valid values. And I'm not 
sure if the website I use to retrieve this will allow this (with 1 
request/sec, I can't imagine they won't see that something is happening).
    
      
Stéphane,

I'm afraid I haven't been following this thread very closely despite the
fact that it is close to my heart.  But I'd assume the terms of use for
the site do not allow you to extract their full database this way and
redistribute it.  Is that not the case?

  
    
By the way, I have a question for geostuff people (Tracey ?) : On the 
geogratis website, there are some files concerning the electoral 
districts ( 
http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169), but 
I don't know anything concerning the format (Arc Export or shape file). 
Is there a way to do something with that ? For example do a process 
what-ever to find out in which electoral district is a point (long-lat) ?
    
      
I can definitely help with this sort of thing if you want to do it.
Geo-data-wrangling is my thing.

Best regards,
  
    


_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca

  

Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Daniel Haran
Maybe there's an algorithm to minimize the number of points you'd have to check?

On 2/8/07, Stéphane Zagar <[hidden email]> wrote:

>
>  I know but I'm not a magician :)
>
>  I don't see any way to get that.
>  My question is to know whether it's useful to be that precise. Maybe my
> view is biaised because I take my examples in Montreal, but a 6 digit postal
> code is a very small polygon. If I just want to match a postal code (a
> point) within a district (an area), it should be possible. No ? Is it an
> ugly way to do it ?
>
>  Steph
>
>
>  Tracey P. Lauriault wrote:
>  the lat long you get is a point, while the 6 digit postal code is a
> polygon! So for the data to be useful you also need the coordinates of
> intersecting lines.
>
>
> Frank Warmerdam wrote:
>
>
>  Stéphane Zagar wrote:
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Tracey P. Lauriault-2
In reply to this post by Stéphane Zagar
you have a series of points that you cannot really tag any useful data
to and do any type of analysis with.  Any assumptions you generate from
such an endeavour  would be spurrious at best.

Electoral data is aggregated according to an electoral boundary file
which i believe is a file we can access, the postal code data is another
way to access that information at a smaller geographical unit which
demonstrates more of the differences at the neighbourhood level (polygon
not point).  Also, as discussed, the postal code boundary files do not
line up with the electoral boundary files. So it is two different kinds
of analysis.

In the end, to make a decision on what is best, we need to know exactly
what questions you want answered and then we choose the best data sets
to do answer that question.

Stéphane Zagar wrote:

> I know but I'm not a magician :)
>
> I don't see any way to get that.
> My question is to know whether it's useful to be that precise. Maybe
> my view is biaised because I take my examples in Montreal, but a 6
> digit postal code is a very small polygon. If I just want to match a
> postal code (a point) within a district (an area), it should be
> possible. No ? Is it an ugly way to do it ?
>
> Steph
>
> Tracey P. Lauriault wrote:
>> the lat long you get is a point, while the 6 digit postal code is a
>> polygon!  So for the data to be useful you also need the coordinates of
>> intersecting lines.
>>
>>
>> Frank Warmerdam wrote:
>>  
>>> Stéphane Zagar wrote:
>>>  
>>>    
>>>> Hey
>>>>
>>>> In fact, technically it's not complicated to get a zip code mapping with
>>>> latitude and longitude :
>>>>
>>>> H3T1A1;45.5113;-73.6162
>>>>    
>>>>      
>>> ...
>>>  
>>>    
>>>> H3T1Z1;45.4977;-73.6257
>>>>
>>>> And I think it's not that much complicated to do the same reverse
>>>> engineering for electoral district.
>>>>
>>>> The problem is that mathematically, there are about 17 millions of valid
>>>> combination of zip code. It take about 1 sec for my script to retrieve
>>>> the zipcode/lat-long so if we do a basic computation it gives us
>>>> about... 100 days to retrieve everything (and probably a lot of
>>>> bandwidth). Even if we set the last number to 1 (what I did in my
>>>> previous test), it's still about 2 millions valid values. And I'm not
>>>> sure if the website I use to retrieve this will allow this (with 1
>>>> request/sec, I can't imagine they won't see that something is happening).
>>>>    
>>>>      
>>> Stéphane,
>>>
>>> I'm afraid I haven't been following this thread very closely despite the
>>> fact that it is close to my heart.  But I'd assume the terms of use for
>>> the site do not allow you to extract their full database this way and
>>> redistribute it.  Is that not the case?
>>>
>>>  
>>>    
>>>> By the way, I have a question for geostuff people (Tracey ?) : On the
>>>> geogratis website, there are some files concerning the electoral
>>>> districts (
>>>> http://geogratis.cgdi.gc.ca/geogratis/en/option/select.do?id=1169), but
>>>> I don't know anything concerning the format (Arc Export or shape file).
>>>> Is there a way to do something with that ? For example do a process
>>>> what-ever to find out in which electoral district is a point (long-lat) ?
>>>>    
>>>>      
>>> I can definitely help with this sort of thing if you want to do it.
>>> Geo-data-wrangling is my thing.
>>>
>>> Best regards,
>>>  
>>>    
>>
>>
>> _______________________________________________
>> CivicAccess-discuss mailing list
>> [hidden email]
>> http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca
>>
>>  
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> CivicAccess-discuss mailing list
> [hidden email]
> http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca
>  



Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Frank Warmerdam
In reply to this post by Stéphane Zagar
Stéphane Zagar wrote:
> Hi Frank,
>
> As someone else on the list said, zip codes and electoral districts are
> a fact, there's no way for them to know how I build my mapping.
 >
> In fact I would be happy to send this mapping to anybody who want to sue
> me with that to see what would be the result. But for the moment, I'm
> not sure if I would be able to get everything since the website I use
> might declare my IP address as fraudulent after only 1000 zip retrieved :p

Stéphane,

It seems clear you take a distinctly different view to this than I do.

To me it is important to ensure clear and legitimate title to my free geodata
as well as software.  I'm not interested in involvement with data obtained
under questionable terms or on the basis that it might be hard to prove that
it was derived contrary to a legal agreement.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org



Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Frank Warmerdam
In reply to this post by Stéphane Zagar
Stéphane Zagar wrote:

> Daniel :
>
> I don't see how this can be copyrighted. Someone (you Daniel ?) told
> that facts can't be copyrighted. I just can't imagine how this could be
> copyrighted ! Damned I could call every and each Canadien citizen and
> ask him his zipcode and his electoral district, then can't use this ?
>
> Concerning the other points you raise :
> - It's amazing that postal codes can overlap several districts... but I
> can't see how to solve this. Even the Election Canada website uses the
> postal code to find your district.
> - I don't see how you get only 1 millions possibilites with this regex.
> You bring the first character from 26 letters to 19. The result is
> 19*10*26*10*26*10 = 12 millions
>
>
> Franck :
>
> I understand you point. However I still don't understand why those data
> are not freely accessible. If they are not available I'll pick them...
> if I can ;)

Stéphane,

I am not claiming the facts are copyright.  I'm afraid I didn't pay attention
earlier in this discussion to see what service you were using and what
terms of use you agreed to in using the service (whether explicit or implicit).

If the terms of use say you can do whatever you want *and* if the service
provider actually had the right to offer this (ie. isn't failing to inforce
some licensing agreement they agreed to for getting the data) then you are
fine.  But I am doubtful.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org



Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Stéphane Zagar
Yes, the service I use specify that the data is copyrighted.... everything unfortunately for this kind of data...

Frank Warmerdam wrote:
Stéphane Zagar wrote:
  
Daniel :

I don't see how this can be copyrighted. Someone (you Daniel ?) told 
that facts can't be copyrighted. I just can't imagine how this could be 
copyrighted ! Damned I could call every and each Canadien citizen and 
ask him his zipcode and his electoral district, then can't use this ?

Concerning the other points you raise :
- It's amazing that postal codes can overlap several districts... but I 
can't see how to solve this. Even the Election Canada website uses the 
postal code to find your district.
- I don't see how you get only 1 millions possibilites with this regex. 
You bring the first character from 26 letters to 19. The result is 
19*10*26*10*26*10 = 12 millions


Franck :

I understand you point. However I still don't understand why those data 
are not freely accessible. If they are not available I'll pick them... 
if I can ;)
    

Stéphane,

I am not claiming the facts are copyright.  I'm afraid I didn't pay attention
earlier in this discussion to see what service you were using and what
terms of use you agreed to in using the service (whether explicit or implicit).

If the terms of use say you can do whatever you want *and* if the service
provider actually had the right to offer this (ie. isn't failing to inforce
some licensing agreement they agreed to for getting the data) then you are
fine.  But I am doubtful.

Best regards,
  

Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Robin Millette
In reply to this post by Tracey P. Lauriault-2
I've been quiet, but busy compiling (thus crashing my browser often -
what's wrong with having 60 tabs open?) a bunch of info and pointers
on database rights in Canada.

On 2/8/07, Tracey P. Lauriault <[hidden email]> wrote:
> Stéphane Zagar wrote:

> > I don't see how this can be copyrighted. Someone (you Daniel ?) told
> > that facts can't be copyrighted. I just can't imagine how this could
> > be copyrighted ! Damned I could call every and each Canadien citizen
> > and ask him his zipcode and his electoral district, then can't use this ?
> it is a unique arrangement of facts.  Most databases are full of facts,
> rivers, households, telephone numbers.  It is not the fact but the
> unique arrangement of these facts that make them copyrightable.

Databases are protected under the Copyright terms in Canada and should
be seen as "compilations".

I'm still going thru all this, but I had to share this quote from a
Legal Study on Databases by  Canadian Heritage:
http://www.canadianheritage.gc.ca/progs/ac-ca/progs/pda-cpb/pubs/database/08_e.cfm

"In the context of databases, data and facts are by themselves not
protectable in favour of the compiler. A compilation is protected
where the selection or arrangement of the content has met the required
test for originality, but only these features of selection or
arrangement qualify for protection. A close similarity between data
and facts of two or more databases could be used to demonstrate that
an unauthorized reproduction of one database by others has occurred.
However, the essential focus must be upon only the selection or
arrangement and not the data and facts themselves."

>From what I could gather, scraping the web for this information and
turning the info into SQL ourselves would be ok with regards to
Copyright, but might be subject to
2. Section 326(1)(b): Theft of Telecommunication Service

See for more info:
http://www.canadianheritage.gc.ca/progs/ac-ca/progs/pda-cpb/pubs/database/21_e.cfm

--
Robin 'oqp' Millette : http://rym.waglo.com/


Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Robin Millette
On 2/9/07, Robin Millette <[hidden email]> wrote:
> I've been quiet, but busy compiling (thus crashing my browser often -
> what's wrong with having 60 tabs open?) a bunch of info and pointers
> on database rights in Canada.

I should have added, I am not a lawyer, this doesn't constitute legal
advice, etc.


Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Stéphane Zagar
In reply to this post by Robin Millette
Interesting point concerning the copyright Robin... Your interpretation of this quote looks correct but with law, nothing is white or black. I wich we have a lawyer here... maybe we could ask to Micheal Geist :)

Tracey :
> In the end, to make a decision on what is best, we need to know exactly 
> what questions you want answered and then we choose the best data sets 
> to do answer that question.

Well... My idea was to take data from Howd they vote and party/MP donors  and sort them locally. For example it would allow someone to type his postal code and know what's the record of his MP in terms of vote, finance campaign, etc. The fact is that people may simply choose their district in a list (but many people don't know the name of the district) and it would be much simpler for us.

My second idea is that I don't understand that this very basic set of data is not free while some others much more complicated are free. (Well I understand that Stats Can want to earn money with it but it's not a reason). So my idea is also to spread a postal code database (in fact I don't know if I would have the guts... and for the moment I don't have such a database). But if this database is not accurate or not usable, then it's not interesting to spend time on that...

Steph


Robin Millette wrote:
I've been quiet, but busy compiling (thus crashing my browser often -
what's wrong with having 60 tabs open?) a bunch of info and pointers
on database rights in Canada.

On 2/8/07, Tracey P. Lauriault [hidden email] wrote:
  
Stéphane Zagar wrote:
    

  
I don't see how this can be copyrighted. Someone (you Daniel ?) told
that facts can't be copyrighted. I just can't imagine how this could
be copyrighted ! Damned I could call every and each Canadien citizen
and ask him his zipcode and his electoral district, then can't use this ?
      
it is a unique arrangement of facts.  Most databases are full of facts,
rivers, households, telephone numbers.  It is not the fact but the
unique arrangement of these facts that make them copyrightable.
    

Databases are protected under the Copyright terms in Canada and should
be seen as "compilations".

I'm still going thru all this, but I had to share this quote from a
Legal Study on Databases by  Canadian Heritage:
http://www.canadianheritage.gc.ca/progs/ac-ca/progs/pda-cpb/pubs/database/08_e.cfm

"In the context of databases, data and facts are by themselves not
protectable in favour of the compiler. A compilation is protected
where the selection or arrangement of the content has met the required
test for originality, but only these features of selection or
arrangement qualify for protection. A close similarity between data
and facts of two or more databases could be used to demonstrate that
an unauthorized reproduction of one database by others has occurred.
However, the essential focus must be upon only the selection or
arrangement and not the data and facts themselves."

>From what I could gather, scraping the web for this information and
turning the info into SQL ourselves would be ok with regards to
Copyright, but might be subject to
2. Section 326(1)(b): Theft of Telecommunication Service

See for more info:
http://www.canadianheritage.gc.ca/progs/ac-ca/progs/pda-cpb/pubs/database/21_e.cfm

  

Reply | Threaded
Open this post in threaded view
|

Re: Zip codes and Electoral districts

Tracey P. Lauriault-2
Stéphane Zagar wrote:

> Interesting point concerning the copyright Robin... Your
> interpretation of this quote looks correct but with law, nothing is
> white or black. I wich we have a lawyer here... maybe we could ask to
> Micheal Geist :)
>
> Tracey :
> > In the end, to make a decision on what is best, we need to know exactly
> > what questions you want answered and then we choose the best data sets
> > to do answer that question.
>
> Well... My idea was to take data from Howd they vote and party/MP
> donors  and sort them locally. For example it would allow someone to
> type his postal code and know what's the record of his MP in terms of
> vote, finance campaign, etc. The fact is that people may simply choose
> their district in a list (but many people don't know the name of the
> district) and it would be much simpler for us.
then the point file is fine for this purpose, until which time they add
new postal codes for new areas of split areas that have become very
densely populated.  But that is an issue anyone has with database
management.  The trick is to ensure that you are as accurate as you can
be and you discuss the limitations of that degree of accuracy.
>
> My second idea is that I don't understand that this very basic set of
> data is not free while some others much more complicated are free.
> (Well I understand that Stats Can want to earn money with it but it's
> not a reason). So my idea is also to spread a postal code database (in
> fact I don't know if I would have the guts... and for the moment I
> don't have such a database). But if this database is not accurate or
> not usable, then it's not interesting to spend time on that...
It is not free because Canada Post a crown corporation used its factual
information that it uses and collects to conduct its business of getting
us the mail.  Then it made a unique arrangement of its facts into a
database which it sells to make money and i guess they would claim to
pay for maintenance.  But also it sells just like statistics canada
sells the data to marketing companies and data resellets who are more
than willing to pay to have a market advantage and use these data for
geodemographic purposes to sell us stuff.  So it seems that in a
democracy citizens have the right to buy and sell stuff, and citizens
have the right to be sold to, but citizens do not have the rights to
their own publicly paid for data to work towards government transparency
and the like.

I would luv for us to deal with that last point somehow, because our
public agencies currently have a monopoly on the data that we as
citizens have paid for.  Even in a market economy this is not fair practice.

Also, it would be really great if all of you who are working on this can
make what your doing and how you are doing it available on the wiki.  
That way we can share the wealth of knowledge being generated and we can
build on it in the future.

>
> Steph
>
>
> Robin Millette wrote:
>> I've been quiet, but busy compiling (thus crashing my browser often -
>> what's wrong with having 60 tabs open?) a bunch of info and pointers
>> on database rights in Canada.
>>
>> On 2/8/07, Tracey P. Lauriault <[hidden email]> wrote:
>>  
>>> Stéphane Zagar wrote:
>>>    
>>
>>  
>>>> I don't see how this can be copyrighted. Someone (you Daniel ?) told
>>>> that facts can't be copyrighted. I just can't imagine how this could
>>>> be copyrighted ! Damned I could call every and each Canadien citizen
>>>> and ask him his zipcode and his electoral district, then can't use this ?
>>>>      
>>> it is a unique arrangement of facts.  Most databases are full of facts,
>>> rivers, households, telephone numbers.  It is not the fact but the
>>> unique arrangement of these facts that make them copyrightable.
>>>    
>>
>> Databases are protected under the Copyright terms in Canada and should
>> be seen as "compilations".
>>
>> I'm still going thru all this, but I had to share this quote from a
>> Legal Study on Databases by  Canadian Heritage:
>> http://www.canadianheritage.gc.ca/progs/ac-ca/progs/pda-cpb/pubs/database/08_e.cfm
>>
>> "In the context of databases, data and facts are by themselves not
>> protectable in favour of the compiler. A compilation is protected
>> where the selection or arrangement of the content has met the required
>> test for originality, but only these features of selection or
>> arrangement qualify for protection. A close similarity between data
>> and facts of two or more databases could be used to demonstrate that
>> an unauthorized reproduction of one database by others has occurred.
>> However, the essential focus must be upon only the selection or
>> arrangement and not the data and facts themselves."
>>
>> >From what I could gather, scraping the web for this information and
>> turning the info into SQL ourselves would be ok with regards to
>> Copyright, but might be subject to
>> 2. Section 326(1)(b): Theft of Telecommunication Service
>>
>> See for more info:
>> http://www.canadianheritage.gc.ca/progs/ac-ca/progs/pda-cpb/pubs/database/21_e.cfm
>>
>>  
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> CivicAccess-discuss mailing list
> [hidden email]
> http://civicaccess.ca/mailman/listinfo/civicaccess-discuss_civicaccess.ca
>