View source for OpenNews/hackdays/insideroutsider

== Knight-Mozilla-MIT "Insider/Outsider" Hack Day ==
The Knight-Mozilla OpenNews project is sponsoring at 24-hour [http://opennews-mit2013.eventbrite.com/ hack day] as a lead-in to the [http://civic.mit.edu/conference2013 2013 MIT-Knight Civic Media Conference]. While the conference is invite-only, the hack day is open to talented developers who want to spend their weekend working with others to build amazing things.

Following the conference theme of "Insider/Outsider," this hack day will be focused on civic data and opening up challenging data sets.

If you're tweeting about this hackday, please use the [https://twitter.com/#!/search/%23datahack #datahack] hashtag.

=== Logistics ===
* <b>Where:</b> MIT Media Lab 5th Floor [https://maps.google.com/maps?q=mit+media+lab&ll=42.361,-71.087412&spn=0.01056,0.017467&oe=utf-8&client=firefox-aurora&channel=fflb&fb=1&gl=us&hq=mit+media+lab&t=v&z=16 75 Amherst Street Cambridge, MA 02139]
* <b>When:</b> 3pm sharp Saturday June 22 to 4pm Sunday June 23
* <b>Will there be food?</b> Yes, there will be food. We will be providing dinner on Saturday night, breakfast and lunch on Sunday, and snacks throughout.
* <b>What to bring</b> You will need to bring your own laptop and power supply. Also, bring any challenging civic data sets you've been wanting to wrangle.
* <b>We'll supply</b> the WiFi, the plugs, and collaboration and brainstorming materials like post-its, sharpies, etc.

==== Schedule ====

<b>Saturday 6/22</b>
* 3pm Opening Circle
* 7pm Dinner
* 10pm Building closes


<b>Sunday 6/23</b>
* 9:00am Building opens
* 9:30am Breakfast
* 12:30pm Lunch
* 2:45pm Show and Tell
* 3:45pm Closing Circle

=== HackDash: Project Teams and Ideas ===

We're going to use [http://onmit.hackdash.org/ HackDash], developed by our friends at Hacks/Hackers Buenos Aires, to help gather teams and ideas this year. [http://bahackaton.herokuapp.com/ Here's an example] from a hackathon in Buenos Aires.

'''How to use HackDash'''
* Go to [http://onmit.hackdash.org/ our HackDash page]. There's an [http://onmit.hackdash.org/p/51a8cdcae8b8537b1100006b example project] listed to show the basic project format.
* To create or join a project, log in with Twitter (if you don't have a Twitter account, [mailto:erika@mozillafoundation.org email Erika] for assistance).
* To create a project:
** Click create a project.
** If your project exists on GitHub, you can import some of the fields from your GitHub repo using the GitHub importer.
** Give your project a title and description. Both of these items will be shown on the project card on the HackDash page. You can also include a photo associated with your project.
** If there's a link associated with your project, you can include that as well as any topical tags.
** The final drop list is for "state" of your project, which you can update as the project progresses from brainstorming to wireframing and so on.
** Click create project!
* Once a project exists, anyone can join, like, or follow the project. The Twitter avatars for team members are displayed at the bottom of the project card.
* Each project card also includes a Disqus comment thread where team members can communicate or other people can offer feedback on the idea.

It should be that simple. We'll have an easy way to collaborate and see all of the projects from the hack day. Please go ahead and start adding project ideas. If a project looks interesting to you, join the team.

[mailto:erika@mozillafoundation.org Let Erika know] if you have any questions or run into issues with the setup. This tool is in active development by [https://github.com/danzajdband/hackdash Dan Zajdband], so we can get help with any questions and any feedback is much appreciated.

=== Data "White Whales" ===

As a way of giving attendees some things to chew on right out from jump, we invited a number of civic data experts to give us lists of their data "white whales"--high value datasets that are currently difficult to access.

'''From Derek Willis, New York Times:'''
* [http://clerk.house.gov/public_disc/foreign/index.aspx U.S. House of Representatives Foreign Travel Reports]
* [http://www.state.gov/r/pa/prs/appt/2013/index.htm U.S. State Department Public Schedule]
* [http://www.whitehouse.gov/omb/oira_meetings/ White House Office of Management and Budget Meeting Records]


'''From Waldo Jacquith, Virginia Decoded:'''
* [http://www.courts.state.va.us/wpcap.htm Supreme Court of Virginia court decisions]
* [http://www.courts.state.va.us/scndex.htm Virginia Court of Appeals court decisions] 

Waldo notes: It's a huge obstacle that I simply haven't put any time into dealing with. Every few months I spend half an hour on trying to put together a system to systematically scrape data out, get discouraged, and give up. Footnotes, blockquotes, and page numbers just kill me, although even if I could get the raw text decently, rendered terribly, I could still extract great metadata from them.

'''From John Keefe & Stephen Menendez, WNYC:''' 
* [https://dl.dropboxusercontent.com/u/6682410/FY%202013%20Schedule%20C%20-%20Merge%20Final1.pdf 2013 New York City Council budget document (warning large PDF download)]
* [http://www.nyc.gov/html/nypd/html/traffic_reports/motor_vehicle_accident_data.shtml NYPD Motor Vehicle Accident Data]

'''From Daniel X O'Neil, Smart Chicago Collaborative/Everyblock:''' 
* [http://www.nyc.gov/html/nypd/html/analysis_and_planning/stop_question_and_frisk_report.shtml NYPD Stop, Question and Frisk Report Database]
The data is amazingly detailed ([http://www.jjay.cuny.edu/web_images/PRIMER_electronic_version.pdf here's a great primer]), and lends itself to great visualizations ([http://www.nytimes.com/interactive/2010/07/11/nyregion/20100711-stop-and-frisk.html?ref=stopandfrisk here's one re: 2009 data]). The data itself is published in a highly inaccessible to regular people (notwithstanding the fact that is extremely well-structured as an SPSS portable file. Publishing this info as an easy-to-search, RSS-ready list of items would be high value.
* [https://apps.health.ny.gov/pdpw/SearchDrugs/Home.action Prescription Drug Prices in New York State]
This is a gem of a lookup tool that cries out for scraping and simple display. The disparity in drug prices is often profound, even in the space of a few blocks. This fits into a general new trend/ huge opportunity to call out disparate health care costs, given the [https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/Medicare-Provider-Charge-Data/Inpatient.html Medicare Provider Charge Data] provided by the U.S. government recently.
* [http://www.lapdonline.org/crime_prevention/content_basic_view/42390 LA Crime Data]
Given the reality of a new mayor in LA, it would be good to look for some data in this enormous city with very little available civic data. It has always bothered me that the LAPD has an exclusive relationship with the LA Times on crime data: [http://www.lapdonline.org/crime_prevention/content_basic_view/42390 http://www.lapdonline.org/crime_prevention/content_basic_view/42390]. Muffing that up might be fun.
* [ftp://66.97.146.93/ Dallas FTP Bulk Crime Database]
This is an enormous, underutilized cache of crime data. Chicago gets lots of attention and plaudits for their crime data, but the Dallas stuff goes even farther back (2000!) and contains narrative that will make your eyes bleed. They have the actual comments typed into the system by actual police officers, including graphic details about horrible crimes and a huge amount of profanity. This is a researcher's treasure chest.

=== Tools & APIs ===

Have tools or APIs that would be helpful for this hack weekend? List them here.

==== [http://tabula.nerdpower.org Tabula] ====

[http://twitter.com/manuelaristaran Manuel Aristarán] and [http://twitter.com/mtigas Mike Tigas] will release Tabula 1.0 on the Hack Day. We will also work on the tool itself. If you want to get your hands dirty with Ruby code and fun table extraction heuristics, feel free to join!

Tabula is split in two components:

* [http://github.com/jazzido/tabula Tabula webapp] — webapp
* [http://github.com/jazzido/tabula-extractor tabula-extractor] — Table extraction and recognition API.


There's a lot of room for improvement in both components. If you feel like helping out, reach out and we'll find you something to work on :)

==== City of Boston Data Portal ====

The City of Boston has several datasets in a [https://data.cityofboston.gov/ Data Portal] and [http://boston.maps.arcgis.com/home/ ArcGIS Server].

You can browse the Data Portal using a [http://mapmeld.github.io/data-portal-treemap D3 Interactive].

Get in touch with [http://twitter.com/mapmeld Nick Doiron] during the hack day for help with Boston data (especially maps).

==== OpenStreetMap extracts ====

Data for the Boston metro area and dozens of other cities are available for download at http://metro.teczno.com/

==== NearbyFYI ====

* [http://said.nearbyfyi.com/docs/V1/ NearbyFYI - local government documents]
* [http://www.nearbyfyi.com Search interface for the documents we are collecting]
What would you do with 100,000+ documents and extracted text from 170 city and town municipalities in Vermont? We collect city and town documents from select board meeting minutes, planning and zoning committees and other local government legislation. These are often published as PDFs and difficult to scrape HTML. We classify, extract entities [People, Companies, Locations], terms and make them searchable. This is a corpus of partially structured raw text from hundreds of cities and towns.

==== Open Data Tech Review by ODI ====
From Marcio Vasconcelos a [https://github.com/theodi/open-data-tech-review/wiki Wiki] compiled by Open Data Institute with several tools.

=== Communications ===
If you're tweeting about this hackday, please use the [https://twitter.com/#!/search/%23datahack #datahack hashtag]

To stay connected to the Knight-Mozilla OpenNews project, [http://www.twitter.com/opennews follow us @opennews] and check the web to find out [http://www.mozillaopennews.org/getinvolved.html additional ways to get involved].

==== Your Blogs ====
If you're blogging about the event, please link to it here.

=== Places to Eat and Drink ===

We'll be feeding you dinner on Saturday and breakfast and lunch on Sunday. Additionally, we'll be providing snacks and plenty of caffeine throughout. That said, if you're looking to find food some other time while you're in town:

* [http://www.yelp.com/biz/za-cambridge  Za] Homemade pickles, excellent wine list.  Rest of menu is pizza and salad and THAT'S IT. Across the street from the Marriott. 
* [http://www.yelp.com/biz/mary-chung-restaurant-cambridge Mary Chung's] Classic MIT hangout.  Chinese food.  Eat the Suan La Chow Show.  In Central Square.  Lip-dragging distance from Le Meridien; an eight minute walk from the Marriott.

'''Do You Want To Eat A Vegetable?'''

[http://www.yelp.com/biz/veggie-galaxy-cambridge Veggie Galaxy] Vegan/vegetarian diner and bakery.  Retro/air-quotes diner food.  Only vegan. 
[http://www.yelp.com/biz/clover-food-lab-cambridge-2  Clover] Vegan restaurant.  Informal, inexpensive.  

'''[http://lifeandcode.tumblr.com/post/10531294713/coming-to-boston-for-ona11-here-are-some-things-to Lisa Williams' Guide to Things To Do, See and Eat in Boston]''' From a native.