Wikipedia talk:WikiProject United States

Data visualization of municipal population changes in Massachusetts

I've put together an animated SVG showing population flows within Massachusetts for each decade between 1850 and 2009, available at http://toolserver.org/~emw/Massachusetts_municipal_population_flows.svg. I've also uploaded it at File:Massachusetts municipal population flows.svg. If anyone reading this uses Chrome, Opera or Firefox 4 (beta), then I would appreciate feedback on how the animation could be improved. I would like to eventually put the animation (possibly ported to .ogv video format) into the article space including any improvements suggested here. Emw (talk) 17:03, 26 February 2011 (UTC)

Wow, impressive. A couple thoughts:

1. There is no indication on whether you are measuring proportional growth or absolute growth. (i.e is solid green +50k people or +100% of previous population?
2. Have municipal borders been unchanged since 1850?
3. The fade-in/fade-out effect seems hard to follow. I think it would be clearer if the fade was shorter and the color was held for some time period. The current year being displayed should appear as text somewhere.
4. The slider bar in the bottom doesn't seem to be synchronized with the color changes. (Chrome 9.0.597.98)
5. Is it possible through some JS scripting to make the slider draggable? I'm not even sure this is possible with the current state of SVG in browsers.

It looks really quite good though. Can you give some background on how you did it? Thanks --ChrisRuvolo (t) 00:30, 1 March 2011 (UTC)

I'll respond piecewise:

1. The color transitions indicate proportional change. That is, they map to the percentage population change of a given municipality. I'll add some bars in the color gradient key to clarify.
2. Municipal boundaries have indeed changed. Gathering and encoding this data will be a significant effort in itself, but thankfully the US censuses indicate when boundaries have changed and sometimes give maps showing exactly what the changes are. It'll probably be necessary to implement this before the animation can reasonably be moved to the article space.

• County boundary changes should probably also be encoded, although it isn't as necessary to complete before moving this particular animation into the article space. If this animation and its associated population count data set were to be scaled to include other states, though, it would be very nice to be able to see less granular population flows, e.g. counties instead of municipalities.

• Another consideration is change of municipal boundaries by virtue of the town or city's dissolution. An example is the municipality of Roxbury and its complete annexation into Boston in 1868. Another different and more unique example is that of Enfield, Massachusetts, which was one of the four entire towns discontinued to make room for the Quabbin Reservoir. Those four towns effectively disappeared from the map: their populations moved elsewhere, and what little land wasn't flooded was annexed to surrounding municipalities. Emw (talk) 05:27, 4 March 2011 (UTC)

1. I'll shorten the duration of the transition. I'm also going to add a version that replaces the slider bar with a prominent year that fades in and out.
2. Noted, will fix.
3. I haven't implemented a prototype, but I'm pretty sure that would be possible. Another useful feature would be tooltips that pop up as the user hovers over a municipality, which would contain the name of the town/city, raw data on population counts for that year, percent change from the previous decade, etc. Unfortunately, Mediawiki automatically removes any scripting embedded in SVGs, so any work here would be futile for the purposes of Wikipedia.

Some background: before making the animation, I spent quite a while gathering population counts for the 351 municipalities in Massachusetts for each decade since 1850. I did this by manually transcribing into a spreadsheet municipal population counts for Massachusetts from scanned PDF copies of US census records: 1850, 1860, 1870, 1890 (also contains 1880 counts), 1920 (also contains 1910 and 1900 counts), 1950 (also contains 1940 and 1930 counts), 1980 (also contains 1970 and 1960 counts), and 1990. Thankfully 2000 and 2009 counts were already in a spreadsheet.

After compiling the raw population count data in a .csv spreadsheet, I took a pre-existing SVG map of Massachusetts municipalities (File:MA_cities_towns.svg) and labeled each SVG path element that corresponded to a municipality. Note that the existing labels in MA_cities_towns.svg had to be removed and replaced with the labels just mentioned; there was otherwise no feasible way to programmatically link the municipal labels that existed in that map to the municipal polygons. For more information on this see here.

I then wrote a Python program to add to each SVG municipality-labeled path a set of SVG animation elements that encoded the color transitions. For parsing and manipulating the XML of the SVG, I used the BeautifulSoup library for Python (which wasn't ideal and made some manual post-processing necessary). Emw (talk) 02:19, 1 March 2011 (UTC)

Scaling up

The method summarized above may be scalable to other US states. Animations for population changes within states could be stitched together to show interesting intra- and inter-regional demographic trends. The underlying population data could also be used by a bot to automatically add to articles on municipalities a detailed table of historical population counts over time (expanding their existing coverage).

I think the main obstacle to this would be gathering decennial population counts for municipalities. Taking the scanned data available in .pdf files available from the US Census Bureau and converting that into programmatically manipulable text is difficult. The task is both extremely tedious, and infeasible to automate given the poor print quality of available scans and the current state of OCR technology.

The best approach would probably be to distribute this massive data entry task among many people. Given the task's tedium, scale, and embarassingly parallel nature it sounds like an ideal project to use Amazon Mechanical Turk. The work could be distributed into "human intelligence tasks" (HITs, as Amazon calls them) where the Mechanical Turk worker (turker) would be provided A) an image of the scanned US Census record for a given decade for a given state and B) a browser-based spreadsheet template listing all municipalities in that state (for that year) with blank fields that the turker would fill in with population counts.

To ensure accuracy, turker-transcribed population counts could be done multiple times for each state/decade and cross-checked against each other and county- and state-level population counts. And turkers could learn more about where there work would end up through a brief text explanation and link back to a Wikipedia page with more information.

I'm not aware of any other Wikipedia-related work that has involved Mechanical Turk. If possible I'd like to get opinions on how feasible this sounds and any problems that seem likely to pop up. Emw (talk) 05:01, 1 March 2011 (UTC)

Kumioko is gone

FYI, Kumioko is gone and so...I guess we just have to go on? --Highspeedrailguy (talk) 15:06, 28 February 2011 (UTC)

Yes, we must soldier on alone, in the darkness, perhaps observing a moment of silence for our fallen leader.--Tomwsulcer (talk) 20:25, 28 February 2011 (UTC)

This is terrible. I haven't seen another editor do more for the US WikiProjects than anyone else. However, I can totally understand being "....ed off" and/or unfairly critiqued by other editors, it's happened to all of us. --Funandtrvl (talk) 20:47, 28 February 2011 (UTC)

Agree fully. I've been there myself. It's something how WP is structured, possibly, that causes battling and frayed nerves? Like football versus ice hockey -- there's usually no ill feelings after football, but in ice hockey, fights break out all the time. Like, Wikipedia is a rink of sorts.--Tomwsulcer (talk) 21:04, 28 February 2011 (UTC)

I am happy to do the housekeeping for the collaboration...well, anyone can do it really. Casliber (talk · contribs) 02:25, 1 March 2011 (UTC)

Thanks everyone and sorry I left you high and dry. I just got tired of the constant drama by many (although not all by any means) editors who are no longer interested in making an encyclopedia. I truly hope that you and the project will continue to thrive and grow. --71.163.243.16 (talk) 01:05, 7 March 2011 (UTC)

Aid for census maintenance

I've made a suggestion that might aid census maintenance at Wikipedia:Village_pump_(proposals)#Use_bots_to_maintain_census_figures. Your comments are welcome. Student7 (talk) 13:21, 1 March 2011 (UTC)

Draft boilerplate for census data

I've put a possible boilerplate for census data on Wikipedia_talk:WikiProject_United_States/proposed census boilerplate. Note that we will change and discuss changes on the same page. Student7 (talk) 19:21, 5 March 2011 (UTC)

@wikimediaus on identi.ca and twitter

I have accounts for @wikimediaus (on identi.ca) and synced with @wikimediaus on Twitter. Perhaps someday there will be a US Wikimedia chapter to use this, but now we have subnational chapters (NYC) and one forming for DC. For now & the future, I would like the account to be used for outreach in the US, highlighting things like US COTM, meetups, etc. If anyone wants to help maintain the account or has suggestions of things to tweet., please let me know. Cheers. --Aude (talk) 18:30, 1 March 2011 (UTC)

Removal of flags

You are invited to join the discussion at Talk:List of Asian American astronauts. RightCowLeftCoast (talk) 12:38, 11 March 2011 (UTC) (Using {{pls}})

template:American English

{{American English}} use of the flag is in question, see Template talk:American English

184.144.160.156 (talk) 06:24, 15 March 2011 (UTC)

request review for state reptiles

Could I please get a project person to review this article? TCO (talk) 08:46, 18 March 2011 (UTC)

What do you mean? Are you trying to ready it for a GA or FA? --Tomwsulcer (talk) 12:19, 18 March 2011 (UTC)