Using the XAPI, I was able to pull just the relevant information from OSM. In this case, I only wanted nodes that contained a highway key with the value of motorway_junction. I used MapQuest‘s XAPI service and quickly generated a URL using a nifty map mashup. I had to manipulate the XAPI URL manually to change the bounding box; New Jersey was too large to search with a statewide bbox, so I broke the state into four portions by changing the latitude ranges. Making smaller requests for the part of the state between latitude 39 and 40, then 40 and 41, allowed my request to go through without timing out.

I was then left with four XML files that contained the highway interchanges. ArcGIS doesn’t know what to do with this, so I whipped up some Python to convert all four tables into one tab-delimited file. This code (included below) can then be used to add the points to ArcMap using the Add XY Data functionality.

All said and done, this project took me about an hour. I was able to send the data off in about 30 minutes, but then spent another thirty minutes cleaning up the script, modifying it to spit out any available key, not just the “ref” tag. The script is GPL, free for you to use as you see fit. Just remember to give credit where it’s due, attributing your data back to OSM.

Taking the time to do this for a friend is a good reminder of why I push our students to do more than just the GIS curriculum. Being familiar with OSM gives you a different point of view on GIS data, schema, and community than what you would gain from most GIS courses and entry-level positions. GIS is already about problem solving; being able to do some basic programming will enable you to tackle much larger problems efficiently. And providing technical assistance to others just makes you a better person.

© John Reiser for new jersey geographer, 2012. | Permalink | No comment | Add to del.icio.us
Post tags: github, mapquest, OpenStreetMap, python, scripts, xapi

Map Maker has a moderation process for all of the edits made by Map Maker users. There are two big issues I see with the manner in which Google has implemented moderation. First, you cannot re-edit (or even touch) new features until they have been moderated. That means that if you are adding a development (or a series of roads on a college campus) you cannot link up the roads that have been previously added and awaiting moderation. The moderation puts a serious crimp in attempts to push out many edits in a small area. Mike Dobson has an incredibly detailed write up on the issues he faced in trying to correct data using Google Map Maker. A very interesting read on how difficult it is to contribute minor fixes.

The second, bigger issue is the moderation itself. Crowdsourced moderation relies on people from potentially anywhere in the world to weigh in on edits. Now, it is likely that most of the edits occurring in the United States are being performed by individuals that are familiar with the area; they’re locals. Locals are the best source of information; they have intimate knowledge of the area and they are able to readily field-verify the information in the system. So we have a pool of local users making edits, but the moderators may not have any prior knowledge on an area in which they moderate.

Here’s a perfect example of the issue with this form of moderation. Google provides an attribute field for buildings that records the construction material. The field is restricted to an enumeration set of four allowable values: “Concrete”, “Mud/Stone”, “Steel” and “Wood”. Several of our campus buildings were held in moderation limbo, as a reviewer felt that my choice of “Steel” was incorrect and should instead be “Concrete.” Why would a reviewer feel the need to question an attribute and hold up the data creation process? Mind you, the values in this field does not impact the manner in which the building is displayed on the map, so it is not critical to know before the feature is drawn. What could a moderator glean from an aerial photograph that would justify questioning the local editor? I know the buildings are constructed using a steel structure and they are faced in brick and stone – I can walk up and touch the building; go inside. I spend hours inside these buildings each work day, but I need to provide a moderator elsewhere some justification or proof to have the edit approved.

Matt drew in Rowan Boulevard Apartments, a building with a rather complex footprint, as the building has a series of setbacks dividing up the residential suites inside. He drew it in Map Maker as close as possible, similar to how it appears in OpenStreetMap. This edit was then altered and poorly generalized. The moderator informed Matt that it should not reflect the roof line. There are no stepbacks and the building is near the nadir of the photo, so the roof line is awfully close to the actual building footprint. He spent a considerable amount of time drawing in a feature to have it simply blown away by some one that likely has never been near the actual structure.

Compare the photo to what Matt was finally able to have approved.

The moderation system in Google Map Maker is broken and will continue to be if local users are not given the benefit of the doubt regarding their edits. OpenStreetMap has the ability to revert vandalism or poorly edited features. Know why people don’t vandalize OpenStreetMap? It’s often a pain in the ass to commit large amounts of vandalism on OSM. Unless you’re a programmer, committing large bogus edits to OSM isn’t easy to do. And on the occasions when vandalism occurs on OSM, it is often corrected by local users. Google’s fear of having bad data on Google Maps (even though so many errors pop up all the time through erroneous business entries scraped from search indexes) is hindering the ability of users to contribute to their platform. Google stole several concepts from OpenStreetMap but not the ones that can elevate it to a platform that could truly produce credible maps from locally-sourced volunteered data. The only thing going for Map Maker is that eventually your edits will appear on the main Google Maps site.

I got involved in Map Maker solely to get Rowan’s campus data on Google Maps. I cannot pull data back out of Map Maker, so after I achieve my goal, I will likely never use Map Maker again. Why deal with all the frustration and reap none of the rewards?

Update: Google Maps routes park visitors down a residential street that doesn’t actually provide access to the park, frustrating the local residents. Also, Google Maps Fail on Tumblr.

© John Reiser for new jersey geographer, 2011. | Permalink | One comment | Add to del.icio.us
Post tags: Google Map Maker, Google Maps, OpenStreetMap

Way, way back in March of 2010, I contacted Google through their Map Content Partners form to submit Rowan University’s GIS data to be included on Google Maps. I heard nothing from them – not even an automated response – for several months. In August of 2010, someone in Administration noticed that Google had relabeled Rowan University as Glassboro State College, a name not used since 1992. I was instructed to try and remedy this as soon as possible. I explained that Google had given me the cold shoulder, but I would try some different avenues. The “report a problem” feature in Google Maps did get “Glassboro State College” off the map, but Rowan remained a featureless blotch within Glassboro.

After giving up on contributing data to Google Maps, I was finally contacted by “Brandon” from the Cities in 3D Team. Brandon requested a sample of the data for review. “Finally!” I thought, “now I can submit our data for Google Maps!” I’ll admit, I got excited. In hindsight, the excitement was premature and irrational. OpenStreetMap has been updated to reflect the campus since June of 2009, when I used my Export to OSM script to dump the facilities data into .osm files. I should be focusing my efforts on a custom, Mapnik-rendered set of OSM tiles that look good on top of aerials, a “Google Maps Hybrid” layer of my own. I can then sandwich in custom tiles instead of overlaying my data on top of the annotation. (Compare anno-on-top to data-on-top.)

Anyway, I eagerly await word back from Google that our data will appear on Google Maps. The notice I receive was far from what I wanted to hear. Instead of accepting the data, I’m instructed that due to the small number of objects to be added, I should use Google Map Maker instead. I was pissed.

I find Google Map Maker to be an incredible disappointment on two major fronts; first, I refused to use the service as I felt Google was in the wrong for keeping VGI-sourced base map data under lock and key. The second, which I will elaborate upon tomorrow, involves the Map Maker tool itself. For a company such as “do no evil” Google to create a framework where users can volunteer time to build its GIS repository with no compensation – not even data retrieval – I find completely unconscionable. Compare their data download form with getting data from OpenStreetMap. To non-techies, OSM’s XML format might be intimidating, but it’s data – no forms to fill out or licenses to sign. Sure, it’s not a shapefile… but you can get them too, without signing any agreement. I loved hearing how a group of OSM mappers hijacked a Google Mapping Party (the concept itself lifted from OSM) and got the assembled group to work on OSM. Many others have asked “why Google, why not OSM?” The common thread to most Map Maker criticism and OpenStreetMap’s major issue with Map Maker is regarding access to the data that has been contributed. Why make data created of the third world (and likely by individuals in the third world) a for-sale product for a first world company? Even within your own neighborhood, why volunteer your time if you can’t reap most of the benefits?

Back from ideology to practicality, I wanted the University on Google, solely because Google is the new Mapquest; it’s the go-to map service for the average person. (Quick aside, Mapquest has Rowan University completely mapped out through its OSM-sourced open.mapquest.com.) Brandon’s earlier suggestion of Map Maker wasn’t acceptable because of my open-access advocacy leanings, it was out of time constraints. “You want me to redraw everything?” That wasn’t going to happen and I let Brandon know:

Brandon,

Thanks for the info. My major concern has been getting an “official” set of data on to Google Maps. When Google switched to their own data and pulled place names from GNIS, Rowan University reverted to Glassboro State College, a name last used in 1992. As “the map guy” for the University, I was asked by several offices within Rowan to “fix Google.” While reporting the antiquated name as a problem did get “Rowan University” restored on Google Maps, it brought the lack of campus data on Google to the attention of many in the University community. It has been frustrating for me, as I have attempted to contact Google to provide official data for use on Maps for a considerable time, while also attempting to squash bugs in the Google Maps data on our campus. Being a University, there have been “creative” additions to the campus on Google Maps, such as the fact that one of our largest academic buildings appeared as a Bed and Breakfast for about a month.

I would like to provide Google with a set of data out of our Facilities GIS DB that is the same source for our printed campus maps. The previously sent data, reformatted to match your requested schema, is from either survey data or from a 3″ planimetric image. The address information included in this layer matches Gloucester County’s 911 data. This is as official as it gets.

Google Map Maker is not a viable option for me. The data for Rowan University that is available through OpenStreetMap  was generated through an export utility that I wrote. It is the same source as I emailed you previously, except that it dates from the summer of 2009. Very little of what is on OSM was hand redrawn. Not only would it be a considerable time sink for me, the tools in Map Maker would not be able to reproduce the high quality data that already exists.

Through the Geovisualization course I teach, we are offering students the opportunity to build SketchUp models of the campus as a final project. Several of the buildings have been completed and I will be uploading them soon. The entire campus is not complete, so if you would like a shapefile with extrusion data, I’d be happy to resubmit the data with those additional attributes.

Again, I would like to thank you for taking the time to respond to me. I understand that the Cities in 3D team might not have the 2D maps as their major focus, but if you could direct my request to the correct group, I would be incredibly grateful. I initially filled out the Data Provider form in March 2010 and have resubmitted the form several times after receiving no response. I even pestered Ed Parsons over Twitter to no avail. So, I am truly appreciative of your assistance in this issue.

Thanks,
John

Brandon responded with two points that made me bristle. First, Rowan was too small for Google: “the number of buildings on campus provide a relatively low priority for Google.” This is a 215 acre campus with 50+ structures, it’s not huge, but it’s also not insignificant. Second, I was encouraged “to do as much as you can on your own, or as part of
your class with the students help.” No one volunteers my cheap labor for projects other than me, buddy.

Through the reconfiguration of our program offerings, Geography and Environmental Studies students are required to have an internship or directed study as part of their capstone research experience. I was able to find a student that really excelled at contributing to OpenStreetMap and asked him if he would be interested in working on updating three map services: Google (via Map Maker), OpenStreetMap (via JOSM) and ESRI’s Community Base Map via (ArcInfo for Advanced Desktops). He agreed and began working on updating Google Maps. I finally bit the bullet, signed into Map Maker and started working with him on some of the edits.

Tomorrow, how Map Maker can only make bad maps.

© John Reiser for new jersey geographer, 2011. | Permalink | 5 comments | Add to del.icio.us
Post tags: Google Map Maker, Google Maps, OpenStreetMap

ArcGIS on the iPhone

ESRI has released ArcGIS for iPhone. It’s now available through the App Store. It’s an iPad/iPhone app, so it should work well on both. I just tried it out on my iPhone, and it’s pretty snappy. The data loads quickly and the interface is clean. There are many layers already available through the app, such as OpenStreetMap.

I like the support for ArcGIS Server, as we will be able to provide our local data to iPhone users. I hope that a later revision will also support WMS, as there are many more free and useful WMS-based services out there than ArcGIS services. Greater support for GPS would be useful. ArcGIS for iPhone can locate you using the embedded GPS, but I don’t believe you can use the GPS to record features. I believe these features will come. James Fee points out that ESRI’s response to the UC FAQ regarding iOS support is positive, claiming that updates will feature editing and geoprocessing. As much as I prefer to edit on a big, widescreen monitor, it would be nice to update my data on the go.

If you have an iPhone or iPad, download it for free from the App Store and check it out.

© John Reiser for new jersey geographer, 2010. | Permalink | One comment | Add to del.icio.us
Post tags: App Store, ArcGIS, ESRI, iPhone, OpenStreetMap

(...)
Read the rest of Student Experiences with OpenStreetMap on my blog.

© John Reiser for new jersey geographer, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: geography, GIS, OpenStreetMap, Teaching

So, during 2009, I had noticed that on OSM, the State of Georgia had land use data. I started to look into how Georgia was so lucky. OSM relies on user contributions, so some savvy user must have added all of those polygons to the map. I contacted that savvy user to find out more. Liber pointed me to some of the methods he and others have used to import GIS data into OpenStreetMap. I was unsatisfied with the existing software, so I looked into the OSM API and wrote my own code to export directly from ArcGIS into the .osm file format.

ExportToOSM.py is my crack at programming an export utility. I wanted something that would export multipolygons from ArcGIS as OSM multipolygon relations and would produce a file free of redundant nodes. I used an earlier version of my script to export the buildings on Rowan’s campus. After fixing a few issues – namely the multipart polygons (take a look at Evergreen Hall, still need to punch in the interior courtyard as a doughnut hole) – I began developing a plan to export NJ’s 2002 Land Use data to OSM.

(...)
Read the rest of Updating OpenStreetMap with Land Use Data on my blog.

© John Reiser for new jersey geographer, 2010. | Permalink | 2 comments | Add to del.icio.us
Post tags: OpenStreetMap, programming, python, scripting

© John Reiser for new jersey geographer, 2009. | Permalink | No comment | Add to del.icio.us
Post tags: OpenStreetMap

I’m also planning a course exercise using OSM. I’m going to encourage students to print out their neighborhood using Walking Papers, recording any updates or fixes, then modifying the OSM data via Potlatch or JOSM. Though it will likely be offered as an extra credit assignment initially, I’m hoping I’ll be able to integrate it and other elements of OSM into my courses in the future.

If you have an interest in mapping or GIS, you should check out OpenStreetMap. Sign up for an account, and start updating in your neighborhood. If you’re in New Jersey, I’ve updated OSM’s wiki with links to the cycle maps to be added, as well as instructions on how to add features to OSM using the State’s 2007 aerial photography. If you’d like to collaborate with me on sharing information via OpenStreetMap, let me know. You can follow me on OpenStreetMap here.

© John Reiser for new jersey geographer, 2009. | Permalink | One comment | Add to del.icio.us
Post tags: assignments, collaboration, courses, cycle maps, cycling, new jersey, OpenStreetMap, Rowan University

Mikel Maron has some more thoughts about OSM and Map Maker, as well as some comparison screenshots. (Via Mapperz)

© John Reiser for new jersey geographer, 2009. | Permalink | One comment | Add to del.icio.us
Post tags: ArcGIS, Google Map Maker, Google Maps, OpenStreetMap, Virtual Earth, Volunteered Information