Census Release - Free but no so accessible data?

also

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
Hi David;

I posted a cleaned up version on Datalibre.ca last week.

http://datalibre.ca/2012/02/08/the-census-is-here-the-census-is-here-sorta/

I will also be posting an FAQ prepared by a government librarian that
merges some of this info and info from a long letter from a statcan
official.

Cheers
t

On Sun, Feb 12, 2012 at 8:25 PM, David Eaves [hidden email] wrote:

Given that StatsCan is now, slowly, beginning to accept that fact that there
data is going to be free, I suspect they are going to me more open to
finding better ways of sharing it. Certainly, the people I know at Treasury
Board which is driving the open data file, are keen on this.

This email is a little long - but a cleaned up version, possible with key
points laid out, or in power point - might be quite helpful. I can think of
a few people that would be open to reading it and using it to lobby for
change. THere are a number of data consumers within government who I think
would strongly agree.

Cheers,
dave

On 12-02-08 8:59 AM, Tracey P. Lauriault wrote:

My speculations have been confirmed.  The data are free but not necessarily
accessible, in the sense that the methods used to disseminating these data
are complicated, unclear and there are some favourite geographies missing -
most notably Dissemination Areas and other are hidden - Census Tracts.

I just went to visit the StatCan site that disseminates the population and
dwelling count data, and discovered that it is open but the data are not
easily discoverable:

For example, if you go to the Census Profile
(http://www12.statcan.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E)
and you want to look up 5 cities at once you cannot!  You can only look up
one city at a time.  Which means you can only download one geography at a
time.  FYI there are over 2000 cities in Canada and if you want to know who
the top 30 are in terms of population, then its "Houston we have a problem!"

Furthermore, once you look at your city, you are provided with CMA, CD,
economic region, electoral districts and population centres.  You are not
provided with census tracts nor dissemination areas, which are smaller
geographies and when you are provided with lets say electoral districts the
data are provided one district at a time and not all districts at one time.
You have to go back and download them one at a time and then assemble the
file.  CT and DA geographies are also not in this list here -
http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/help-aide/H05.cfm?Lang=E.

To get to CTs (no DAs to be found) my friend Sara a GIS expert at the Social
Planning and Research Council of Hamilton found this method:

Go here:

http://www12.statcan.ca/census-recensement/2011/rt-td/index-eng.cfm

Click on Thematic  Maps (scroll down), go to CMA maps, choose your
location. Then on the following page there will be a link to the map and a
table with all the pop change values for each CT.

Alternatively, and again thanks to Sara you can do the following:

Go here:

http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E&TABID=3#tab3

Then type in a random CT (you can use the example given at the bottom of
the list).

On the next page, click the CT number

On the next page, click the download tab. Then scroll to Option 2, and
select Census tracts and your data format, and “Continue” – Voila, it will
download a file for population counts for all CTs in Canada!

This is absolutely absurd.  Again, what if you want 5 cities, and you want
the CTs not by CMA but by CD and CSD, which is what most people want as the
CMA is not an administrative geography it is a construct of StatCan, and it
is normally larger than a CSD and CDs and includes some cities and not
others, and some cities fall into two CMAs. CDs and CSD correspond to a
city's or municipality's which actual administrative and legal boundaries.

Ted, the GIS expert at Community Development Halton, who was trying to join
the CT data with his geomatics discovered the following:

Unfortunately, the CT table is a mess for GIS purposes. For each CT, there
are 7 entries (rows) for each discrete piece of information (Population in
2006, 2006 to 2011 population change (%), Total private dwellings, Private
dwellings occupied by usual residents, Population density per square
kilometre, Land area (square km)).  When trying to perform a join, ArcGIS
doesn’t know which of the rows to join on to map it.

You can however, download complete files for the country here
http://www12.statcan.ca/census-recensement/2011/dp-pd/prof/details/page_Download-Telecharger.cfm?Lang=E&Tab=2&Geo1=CT&Code1=0880&Geo2=CMA&Code2=505&Data=Count&SearchText=5050045.00&SearchType=Begins&SearchPR=01&B1=All&Custom=&TABID=3
selecting by selecting Option 2 – Comprehensive download file for a selected
geographic level.

DAs remain missing, and if you download the CT file they are organized only
by CMA, you do not have a way to know which are in your CD or CSD and that
would be a nice addition.  Even better would be a table that provided CTs,
and city, or electoral districts and the CTs they contain and CSD with their
postal codes, CTs or CSDs and their DAs and so on.

Analyst will do fine with this release, after incessant digging, but the GIS
folks will have to play around with things and they will grumble at the
waste of time incurred with coding and joining.  Journalists and the public
however, will not be able to know what to do if they want to compare
cities.  People default mistakenly to CMAs, but CMAs a city they are not.

Sara also pointed me to this one - Population and Dwelling Count Highlight
Tables, 2011 Census, which was in all fairness not easy to find, but it has
tables comparing CSDs.
http://www12.statcan.ca/census-recensement/2011/dp-pd/hlt-fst/pd-pl/Index-eng.cfm?Lang=Eng.
This will help most people.

These reference maps are also excellent -
http://www.statcan.gc.ca/pub/92-196-x/2011001/maps-cartes-eng.htm, these
help unravel geo references.  You can also download geographic files here -
http://www12.statcan.ca/census-recensement/2011/geo/bound-limit/bound-limit-eng.cfm.
Again - no DAs

The search by postal code is a nice feature, as finally you can enter your
postal code and find out which census geographies you fall into.
Interestingly, dissemination areas are not a part of that list.
(http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E&TABID=2#tab2).
People however really want that postal code file available to them for
free!  If the government is going open data then one would presume Crown
Corporations and Agencies are part of that deal.

However, what if you want all the postal codes for your city, or all the CTs
and DAs for you city and what if you want that for more than one city at a
time, then you are out of luck as the tool does not allow for that type of
access.

Anyway, there will no doubt be more discoveries and grumblings and I hope
StatCan will work with users to make these things more useable.

Finally, a community of practice is really important, the SPNO data list for
instance, was busy this morning with analysts communicating with each other
as they were looking for things.  These folks know their stuff well and have
their members in their communities to answer to, who will no doubt be
looking for NEW DATA arranged in a way that is meaningful and that
communicates to them.  Social Planning and Community Development councils
have been working with these data for a very long time have much of
expertise.  Demographic and geographic data are complicated and you need to
know how to work with them and do so with care.

Perhaps, as David E.  pointed out, StatCan will begin training people more
broadly on how to use these data!  Alternatively, people may find a way to
better resource planning councils so that they can train journalists and
others on how to work with these data on StatCan's behalf.

Cheers
t
--
Tracey P. Lauriault
613-234-2805

"Every epoch dreams the one that follows it's the dream form of the future,
not its reality" it is the "wish image of the collective".

Walter Benjamin, between 1927-1940,
(http://www.columbia.edu/itc/architecture/ockman/pdfs/dossier_4/buck-morss.pdf)

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

I disagree: This is not a thing government does well.
I think they should put all of their data on a publicly available ftp
site with accompanying metadata and let any number of sites spring up
to properly searchablize (made that one up!) / present / visualize  /
distribute the data....

They should also  (I can't believe they don't do this now!) have
cryptographic hashes for their data, with a URL in their metadata for
the hash file, so I can download their data from a 3rd party site and
verify it has not been modified by comparing it to the hash on the gov
site. SHA-2 512 is good enough for now. SHA-3 when it is out..

https://en.wikipedia.org/wiki/Cryptographic_hash_function
https://en.wikipedia.org/wiki/SHA-2

-Glen Newton

On Tue, Feb 14, 2012 at 7:14 PM, David Eaves [hidden email] wrote:

I actually think all these problems would be solved in the government simply
hired an org like Socrata to develop a real portal for them. It tends to
have all the features you describe.

As the pilot continues, we need to push them on this front. Totally agree.

D

On 12-02-14 2:12 PM, Tracey P. Lauriault wrote:

Excellent David!

My usual concern is a lack of consultation with expert users.  It is
true that these data can be found on the opendata.gc.ca portal, but
their search limitations are in fact worse than those of StatCan when
it comes to census data as there is no real way to narrow the search
by topic or geography or to select multiple geographies.  Please do
not interpret this as being ungrateful for what they have done nor to
the kindnes you demonstrated by forwarding this along.  While StatCan
has issues, they do know their data and they wrap them in a context,
and this is important.  The TBS portal on the other hand treats all
data types in the same way, even though there are better ways to
access certain types of data and there is a huge difference between
science data, geomatics data, census data, transparency data, etc.

I would like to see something between the Discovery Portal
(http://geodiscover.cgdi.ca/web/guest/browse-catalog - they sadly no
longer have the find data via a map function available) and how
StatCan does it.  I think there also needs to be a recognition that
there will never be one model to fit all data types which I think is
what the TBS portal is trying to do. This is also and issue I think
may city open data portals will begin to see as they scale and
diversify.  I would hate to see TBS force a direction upon all fed gov
data providers to disseminate their data in the same way they do.  I
think some metadata fields that allow for federation is good, and that
is how the TBS portal I hope is federating data, but communities of
practice have evolved, in some cases over centuries (Astronomers,
geographers, statisticians, demographers), and to strip those data
sets of context is I believe a big mistake, analogous to how we have
lost subject matter specialists in Archives and now no one knows how
to find anything, especially at LAC.  Some sort of interoperability is
for sure required, but in some respects, I would almost prefer to see
the TBS point to existing portals, although, its system and pointing
to existing portals would be optimum.  Here is an example of what I
mean -
http://gcmd.gsfc.nasa.gov/KeywordSearch/Home.do?Portal=amd&MetadataType=0
and you will find many of these types of systems in the sciences and
geomatics.

The following is an FAQ produced by a govdoc expert related to the
Statcan issue-
http://datalibre.ca/2012/02/13/guest-blog-post-census-data-liberated-faq-by-aspi-balsara-government-documents-librarian/

Cheers
t

On Tue, Feb 14, 2012 at 4:23 PM, David Eaves[hidden email]  wrote:

Forwarded from correspondence from the Government's Open Data team:

In response to the concerns raised in Tracey Lauriault’s blog, we have
searched through the Open Data portal and can provide you with the
following
information that may help address some of the concerns.

The census profile data is available on the Open Data portal with access
to
all the different geography levels made available by Statistics Canada.
You
can download the complete file of census profile data at a selected
geography level (e.g. federal electoral districts, economic regions)
including the census tracts geography level. Statistics Canada has not
yet
made available a similar file broken down by Dissemination Area (DA). As
such, the DA level data is not available on the Open Data Portal.

Statistics Canada has been contacted to inquire about their plans to
include
the DA level data as well as data that would enable the cross-referencing
of
geography levels (e.g. census tracts, cities, federal electoral
districts,
etc.) on the Open Data portal.

So...  not a perfect answer but at least it is one their radar. My sense
is
the pressure needs to be on StatsCan to rethink how they share data -
something I know we are all keen on and have been active in. This doesn't
resolve the issue, but hopefully is helpful information for us advocates.

Dave

On 12-02-12 6:33 PM, Tracey P. Lauriault wrote:

Hi David;
I posted a cleaned up version on Datalibre.ca last week.

http://datalibre.ca/2012/02/08/the-census-is-here-the-census-is-here-sorta/
I will also be posting an FAQ prepared by a government librarian that
merges some of this info and info from a long letter from a statcan
official.
Cheers
t
On Sun, Feb 12, 2012 at 8:25 PM, David Eaves[hidden email]  wrote:

Given that StatsCan is now, slowly, beginning to accept that fact that
there
data is going to be free, I suspect they are going to me more open to
finding better ways of sharing it. Certainly, the people I know at
Treasury
Board which is driving the open data file, are keen on this.
This email is a little long - but a cleaned up version, possible with key
points laid out, or in power point - might be quite helpful. I can think
of
a few people that would be open to reading it and using it to lobby for
change. THere are a number of data consumers within government who I
think
would strongly agree.
Cheers,
dave
On 12-02-08 8:59 AM, Tracey P. Lauriault wrote:
My speculations have been confirmed.  The data are free but not
necessarily
accessible, in the sense that the methods used to disseminating these
data
are complicated, unclear and there are some favourite geographies missing
-
most notably Dissemination Areas and other are hidden - Census Tracts.
I just went to visit the StatCan site that disseminates the population
and
dwelling count data, and discovered that it is open but the data are not
easily discoverable:
For example, if you go to the Census Profile

(http://www12.statcan.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E)
and you want to look up 5 cities at once you cannot!  You can only look
up
one city at a time.  Which means you can only download one geography at a
time.  FYI there are over 2000 cities in Canada and if you want to know
who
the top 30 are in terms of population, then its "Houston we have a
problem!"
Furthermore, once you look at your city, you are provided with CMA, CD,
economic region, electoral districts and population centres.  You are not
provided with census tracts nor dissemination areas, which are smaller
geographies and when you are provided with lets say electoral districts
the
data are provided one district at a time and not all districts at one
time.
You have to go back and download them one at a time and then assemble the
file.  CT and DA geographies are also not in this list here -

http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/help-aide/H05.cfm?Lang=E.
To get to CTs (no DAs to be found) my friend Sara a GIS expert at the
Social
Planning and Research Council of Hamilton found this method:

Go here:
http://www12.statcan.ca/census-recensement/2011/rt-td/index-eng.cfm
Click on Thematic  Maps (scroll down), go to CMA maps, choose your
location. Then on the following page there will be a link to the map and
a
table with all the pop change values for each CT.

Alternatively, and again thanks to Sara you can do the following:

Go here:

http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E&TABID=3#tab3
Then type in a random CT (you can use the example given at the bottom of
the list).
On the next page, click the CT number
On the next page, click the download tab. Then scroll to Option 2, and
select Census tracts and your data format, and “Continue” – Voila, it
will
download a file for population counts for all CTs in Canada!

This is absolutely absurd.  Again, what if you want 5 cities, and you
want
the CTs not by CMA but by CD and CSD, which is what most people want as
the
CMA is not an administrative geography it is a construct of StatCan, and
it
is normally larger than a CSD and CDs and includes some cities and not
others, and some cities fall into two CMAs. CDs and CSD correspond to a
city's or municipality's which actual administrative and legal
boundaries.
Ted, the GIS expert at Community Development Halton, who was trying to
join
the CT data with his geomatics discovered the following:

Unfortunately, the CT table is a mess for GIS purposes. For each CT,
there
are 7 entries (rows) for each discrete piece of information (Population
in
2006, 2006 to 2011 population change (%), Total private dwellings,
Private
dwellings occupied by usual residents, Population density per square
kilometre, Land area (square km)).  When trying to perform a join, ArcGIS
doesn’t know which of the rows to join on to map it.

You can however, download complete files for the country here

http://www12.statcan.ca/census-recensement/2011/dp-pd/prof/details/page_Download-Telecharger.cfm?Lang=E&Tab=2&Geo1=CT&Code1=0880&Geo2=CMA&Code2=505&Data=Count&SearchText=5050045.00&SearchType=Begins&SearchPR=01&B1=All&Custom=&TABID=3
selecting by selecting Option 2 – Comprehensive download file for a
selected
geographic level.
DAs remain missing, and if you download the CT file they are organized
only
by CMA, you do not have a way to know which are in your CD or CSD and
that
would be a nice addition.  Even better would be a table that provided
CTs,
and city, or electoral districts and the CTs they contain and CSD with
their
postal codes, CTs or CSDs and their DAs and so on.
Analyst will do fine with this release, after incessant digging, but the
GIS
folks will have to play around with things and they will grumble at the
waste of time incurred with coding and joining.  Journalists and the
public
however, will not be able to know what to do if they want to compare
cities.  People default mistakenly to CMAs, but CMAs a city they are not.
Sara also pointed me to this one - Population and Dwelling Count
Highlight
Tables, 2011 Census, which was in all fairness not easy to find, but it
has
tables comparing CSDs.

http://www12.statcan.ca/census-recensement/2011/dp-pd/hlt-fst/pd-pl/Index-eng.cfm?Lang=Eng.
This will help most people.
These reference maps are also excellent -
http://www.statcan.gc.ca/pub/92-196-x/2011001/maps-cartes-eng.htm, these
help unravel geo references.  You can also download geographic files here
-

http://www12.statcan.ca/census-recensement/2011/geo/bound-limit/bound-limit-eng.cfm.
Again - no DAs
The search by postal code is a nice feature, as finally you can enter
your
postal code and find out which census geographies you fall into.
Interestingly, dissemination areas are not a part of that list.

(http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E&TABID=2#tab2).
People however really want that postal code file available to them for
free!  If the government is going open data then one would presume Crown
Corporations and Agencies are part of that deal.
However, what if you want all the postal codes for your city, or all the
CTs
and DAs for you city and what if you want that for more than one city at
a
time, then you are out of luck as the tool does not allow for that type
of
access.
Anyway, there will no doubt be more discoveries and grumblings and I hope
StatCan will work with users to make these things more useable.
Finally, a community of practice is really important, the SPNO data list
for
instance, was busy this morning with analysts communicating with each
other
as they were looking for things.  These folks know their stuff well and
have
their members in their communities to answer to, who will no doubt be
looking for NEW DATA arranged in a way that is meaningful and that
communicates to them.  Social Planning and Community Development councils
have been working with these data for a very long time have much of
expertise.  Demographic and geographic data are complicated and you need
to
know how to work with them and do so with care.
Perhaps, as David E.  pointed out, StatCan will begin training people
more
broadly on how to use these data!  Alternatively, people may find a way
to
better resource planning councils so that they can train journalists and
others on how to work with these data on StatCan's behalf.
Cheers
t
--
Tracey P. Lauriault
613-234-2805
"Every epoch dreams the one that follows it's the dream form of the
future,
not its reality" it is the "wish image of the collective".
Walter Benjamin, between 1927-1940,

(http://www.columbia.edu/itc/architecture/ockman/pdfs/dossier_4/buck-morss.pdf)
_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

Thanks David;

BTW DLI is 1996!  My mistake, thanks MF.  At least I did not say 1886!

David, I looked at the Chicago and the US sites and unless I am
missing something, these are not the best search tools I have ever
seen and it is very hard to narrow things down.  I would most
definitely not want to dig thoughts lists of thousands of data sets
that way.  Again, the context is missing.  I want something as
powerful as a library catalog with multiple ways to search for content
and way more structure.

I would hate to search for books the way data are searched for in
either of these.

The tagging system is also off, search health and see what you get.
Also there is no way to narrow the search by geographic unit, etc.

Am I missing something?  This is not at all like the British Antarctic
Survey example I sent.

On Tue, Feb 14, 2012 at 11:39 PM, David Eaves [hidden email] wrote:

No prob, here are some:

Data.gov
Chicago's open data portal
State of Oregon
Kenya's open data portal

On 2012-02-14, at 8:26 PM, "Tracey P. Lauriault" [hidden email] wrote:

Before going in either direction, it would great to talk with users
and I would also like to hear more from people who work with and have
managed big heterogeneous data sets like the data librarians, the data
archivists and some of the geomatics folks.

The DLI started as an ftp site like you suggested Glen, back in 1986,
they could do it as they were the only people who knew the data sets
well enough and it was one of the few ways available at the time to do
so.  They have moved on to develop tools like ODESI.  The Community
Data Program also did that at the beginning, but it did not work for
most of our community users as they really wanted to fish through
descriptions and have the content organized in a way they understood,
we had people pay to join the consortium but never used the data as
they just did not know how to. It was not the best thing.

Also, I wonder how many social worker coding hotshots there are who
could work on some of the data community groups are interested in, and
I wonder who would come and develop discovery tools for those kinds of
essential but not sexy data sets, so those communities would probably
remain under served as we do not get to see too many engineers
gravitate to those topic areas and wanting to create simple UIs!  GPS
transit files seem to get them all going though.  In addition, you
would have to be an expert at all of the different kinds of files, and
many people are not, and you still need catalogers to describe the
stuff so...

David, can you point me to socrata sites in big institutions
delivering lots of complex heterogeneous data?

Cheers
t

On Tue, Feb 14, 2012 at 8:09 PM, David Eaves [hidden email] wrote:

Definitely agree with the second paragraph.

Re the first, I strongly disagree. This has been tried for the last decade.
It is great for a small number of particularly tech savvy users. Not so
helpful for everyone else.

More importantly, such an approach creates all sorts of barriers and
marginalizes a number of users. There is no guarantee such sites with a)
exist at all. b) will index all the data, they may just focus on data that
those who are willing to pay them to organize are looking for. This will
help the wealthier companies and organizations, but would leave more
marginalized communities - those with less capacity or resources - out in
the cold.

I understand the small government, libertarian appeal of the approach - and
there is no reason why the government couldn't also do that. But given how
cheap a socrata implementation is, it feels like the right thing to do.

Dave

On 12-02-14 5:00 PM, Glen Newton wrote:

I disagree: This is not a thing government does well.
I think they should put all of their data on a publicly available ftp
site with accompanying metadata and let any number of sites spring up
to properly searchablize (made that one up!) / present / visualize  /
distribute the data....

They should also  (I can't believe they don't do this now!) have
cryptographic hashes for their data, with a URL in their metadata for
the hash file, so I can download their data from a 3rd party site and
verify it has not been modified by comparing it to the hash on the gov
site. SHA-2 512 is good enough for now. SHA-3 when it is out..

https://en.wikipedia.org/wiki/Cryptographic_hash_function
https://en.wikipedia.org/wiki/SHA-2

-Glen Newton

On Tue, Feb 14, 2012 at 7:14 PM, David Eaves [hidden email] wrote:

I actually think all these problems would be solved in the government simply
hired an org like Socrata to develop a real portal for them. It tends to
have all the features you describe.

As the pilot continues, we need to push them on this front. Totally agree.

D

On 12-02-14 2:12 PM, Tracey P. Lauriault wrote:

Excellent David!

My usual concern is a lack of consultation with expert users.  It is
true that these data can be found on the opendata.gc.ca portal, but
their search limitations are in fact worse than those of StatCan when
it comes to census data as there is no real way to narrow the search
by topic or geography or to select multiple geographies.  Please do
not interpret this as being ungrateful for what they have done nor to
the kindnes you demonstrated by forwarding this along.  While StatCan
has issues, they do know their data and they wrap them in a context,
and this is important.  The TBS portal on the other hand treats all
data types in the same way, even though there are better ways to
access certain types of data and there is a huge difference between
science data, geomatics data, census data, transparency data, etc.

I would like to see something between the Discovery Portal
(http://geodiscover.cgdi.ca/web/guest/browse-catalog - they sadly no
longer have the find data via a map function available) and how
StatCan does it.  I think there also needs to be a recognition that
there will never be one model to fit all data types which I think is
what the TBS portal is trying to do. This is also and issue I think
may city open data portals will begin to see as they scale and
diversify.  I would hate to see TBS force a direction upon all fed gov
data providers to disseminate their data in the same way they do.  I
think some metadata fields that allow for federation is good, and that
is how the TBS portal I hope is federating data, but communities of
practice have evolved, in some cases over centuries (Astronomers,
geographers, statisticians, demographers), and to strip those data
sets of context is I believe a big mistake, analogous to how we have
lost subject matter specialists in Archives and now no one knows how
to find anything, especially at LAC.  Some sort of interoperability is
for sure required, but in some respects, I would almost prefer to see
the TBS point to existing portals, although, its system and pointing
to existing portals would be optimum.  Here is an example of what I
mean -
http://gcmd.gsfc.nasa.gov/KeywordSearch/Home.do?Portal=amd&MetadataType=0
and you will find many of these types of systems in the sciences and
geomatics.

The following is an FAQ produced by a govdoc expert related to the
Statcan issue-
http://datalibre.ca/2012/02/13/guest-blog-post-census-data-liberated-faq-by-aspi-balsara-government-documents-librarian/

Cheers
t

On Tue, Feb 14, 2012 at 4:23 PM, David Eaves[hidden email]  wrote:

Forwarded from correspondence from the Government's Open Data team:

In response to the concerns raised in Tracey Lauriault’s blog, we have
searched through the Open Data portal and can provide you with the
following
information that may help address some of the concerns.

The census profile data is available on the Open Data portal with access
to
all the different geography levels made available by Statistics Canada.
You
can download the complete file of census profile data at a selected
geography level (e.g. federal electoral districts, economic regions)
including the census tracts geography level. Statistics Canada has not
yet
made available a similar file broken down by Dissemination Area (DA). As
such, the DA level data is not available on the Open Data Portal.

Statistics Canada has been contacted to inquire about their plans to
include
the DA level data as well as data that would enable the cross-referencing
of
geography levels (e.g. census tracts, cities, federal electoral
districts,
etc.) on the Open Data portal.

So...  not a perfect answer but at least it is one their radar. My sense
is
the pressure needs to be on StatsCan to rethink how they share data -
something I know we are all keen on and have been active in. This doesn't
resolve the issue, but hopefully is helpful information for us advocates.

Dave

On 12-02-12 6:33 PM, Tracey P. Lauriault wrote:

Hi David;
I posted a cleaned up version on Datalibre.ca last week.

http://datalibre.ca/2012/02/08/the-census-is-here-the-census-is-here-sorta/
I will also be posting an FAQ prepared by a government librarian that
merges some of this info and info from a long letter from a statcan
official.
Cheers
t
On Sun, Feb 12, 2012 at 8:25 PM, David Eaves[hidden email]  wrote:

Given that StatsCan is now, slowly, beginning to accept that fact that
there
data is going to be free, I suspect they are going to me more open to
finding better ways of sharing it. Certainly, the people I know at
Treasury
Board which is driving the open data file, are keen on this.
This email is a little long - but a cleaned up version, possible with key
points laid out, or in power point - might be quite helpful. I can think
of
a few people that would be open to reading it and using it to lobby for
change. THere are a number of data consumers within government who I
think
would strongly agree.
Cheers,
dave
On 12-02-08 8:59 AM, Tracey P. Lauriault wrote:
My speculations have been confirmed.  The data are free but not
necessarily
accessible, in the sense that the methods used to disseminating these
data
are complicated, unclear and there are some favourite geographies missing
-
most notably Dissemination Areas and other are hidden - Census Tracts.
I just went to visit the StatCan site that disseminates the population
and
dwelling count data, and discovered that it is open but the data are not
easily discoverable:
For example, if you go to the Census Profile

(http://www12.statcan.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E)
and you want to look up 5 cities at once you cannot!  You can only look
up
one city at a time.  Which means you can only download one geography at a
time.  FYI there are over 2000 cities in Canada and if you want to know
who
the top 30 are in terms of population, then its "Houston we have a
problem!"
Furthermore, once you look at your city, you are provided with CMA, CD,
economic region, electoral districts and population centres.  You are not
provided with census tracts nor dissemination areas, which are smaller
geographies and when you are provided with lets say electoral districts
the
data are provided one district at a time and not all districts at one
time.
You have to go back and download them one at a time and then assemble the
file.  CT and DA geographies are also not in this list here -

http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/help-aide/H05.cfm?Lang=E.
To get to CTs (no DAs to be found) my friend Sara a GIS expert at the
Social
Planning and Research Council of Hamilton found this method:

Go here:
http://www12.statcan.ca/census-recensement/2011/rt-td/index-eng.cfm
Click on Thematic  Maps (scroll down), go to CMA maps, choose your
location. Then on the following page there will be a link to the map and
a
table with all the pop change values for each CT.

Alternatively, and again thanks to Sara you can do the following:

Go here:

http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E&TABID=3#tab3
Then type in a random CT (you can use the example given at the bottom of
the list).
On the next page, click the CT number
On the next page, click the download tab. Then scroll to Option 2, and
select Census tracts and your data format, and “Continue” – Voila, it
will
download a file for population counts for all CTs in Canada!

This is absolutely absurd.  Again, what if you want 5 cities, and you
want
the CTs not by CMA but by CD and CSD, which is what most people want as
the
CMA is not an administrative geography it is a construct of StatCan, and
it
is normally larger than a CSD and CDs and includes some cities and not
others, and some cities fall into two CMAs. CDs and CSD correspond to a
city's or municipality's which actual administrative and legal
boundaries.
Ted, the GIS expert at Community Development Halton, who was trying to
join
the CT data with his geomatics discovered the following:

Unfortunately, the CT table is a mess for GIS purposes. For each CT,
there
are 7 entries (rows) for each discrete piece of information (Population
in
2006, 2006 to 2011 population change (%), Total private dwellings,
Private
dwellings occupied by usual residents, Population density per square
kilometre, Land area (square km)).  When trying to perform a join, ArcGIS
doesn’t know which of the rows to join on to map it.

You can however, download complete files for the country here

http://www12.statcan.ca/census-recensement/2011/dp-pd/prof/details/page_Download-Telecharger.cfm?Lang=E&Tab=2&Geo1=CT&Code1=0880&Geo2=CMA&Code2=505&Data=Count&SearchText=5050045.00&SearchType=Begins&SearchPR=01&B1=All&Custom=&TABID=3
selecting by selecting Option 2 – Comprehensive download file for a
selected
geographic level.
DAs remain missing, and if you download the CT file they are organized
only
by CMA, you do not have a way to know which are in your CD or CSD and
that
would be a nice addition.  Even better would be a table that provided
CTs,
and city, or electoral districts and the CTs they contain and CSD with
their
postal codes, CTs or CSDs and their DAs and so on.
Analyst will do fine with this release, after incessant digging, but the
GIS
folks will have to play around with things and they will grumble at the
waste of time incurred with coding and joining.  Journalists and the
public
however, will not be able to know what to do if they want to compare
cities.  People default mistakenly to CMAs, but CMAs a city they are not.
Sara also pointed me to this one - Population and Dwelling Count
Highlight
Tables, 2011 Census, which was in all fairness not easy to find, but it
has
tables comparing CSDs.

http://www12.statcan.ca/census-recensement/2011/dp-pd/hlt-fst/pd-pl/Index-eng.cfm?Lang=Eng.
This will help most people.
These reference maps are also excellent -
http://www.statcan.gc.ca/pub/92-196-x/2011001/maps-cartes-eng.htm, these
help unravel geo references.  You can also download geographic files here
-

http://www12.statcan.ca/census-recensement/2011/geo/bound-limit/bound-limit-eng.cfm.
Again - no DAs
The search by postal code is a nice feature, as finally you can enter
your
postal code and find out which census geographies you fall into.
Interestingly, dissemination areas are not a part of that list.

(http://www12.statcan.gc.ca/census-recensement/2011/dp-pd/prof/index.cfm?Lang=E&TABID=2#tab2).
People however really want that postal code file available to them for
free!  If the government is going open data then one would presume Crown
Corporations and Agencies are part of that deal.
However, what if you want all the postal codes for your city, or all the
CTs
and DAs for you city and what if you want that for more than one city at
a
time, then you are out of luck as the tool does not allow for that type
of
access.
Anyway, there will no doubt be more discoveries and grumblings and I hope
StatCan will work with users to make these things more useable.
Finally, a community of practice is really important, the SPNO data list
for
instance, was busy this morning with analysts communicating with each
other
as they were looking for things.  These folks know their stuff well and
have
their members in their communities to answer to, who will no doubt be
looking for NEW DATA arranged in a way that is meaningful and that
communicates to them.  Social Planning and Community Development councils
have been working with these data for a very long time have much of
expertise.  Demographic and geographic data are complicated and you need
to
know how to work with them and do so with care.
Perhaps, as David E.  pointed out, StatCan will begin training people
more
broadly on how to use these data!  Alternatively, people may find a way
to
better resource planning councils so that they can train journalists and
others on how to work with these data on StatCan's behalf.
Cheers
t
--
Tracey P. Lauriault
613-234-2805
"Every epoch dreams the one that follows it's the dream form of the
future,
not its reality" it is the "wish image of the collective".
Walter Benjamin, between 1927-1940,

(http://www.columbia.edu/itc/architecture/ockman/pdfs/dossier_4/buck-morss.pdf)
_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss
_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

--
Tracey P. Lauriault
613-234-2805

"Every epoch dreams the one that follows it's the dream form of the
future, not its reality" it is the "wish image of the collective".

Walter Benjamin, between 1927-1940,
(http://www.columbia.edu/itc/architecture/ockman/pdfs/dossier_4/buck-morss.pdf)
_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

_______________________________________________
CivicAccess-discuss mailing list
[hidden email]
http://lists.pwd.ca/mailman/listinfo/civicaccess-discuss

Census Release - Free but no so accessible data?

Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?

Re: Census Release - Free but no so accessible data?