Bizarre GeoServer WMS rendering possibly due to ArcGIS handling of PostGIS data

I just submitted a problem to the OpenGeo support forum regarding an unusual bug I’ve discovered in GeoServer. Multipart polygon geometries are producing “unintentional art” when rendered through GeoServer.

NJ Landscape Project data improperly rendered.

Several GIS data sets that have been brought into PostGIS using ArcGIS 10.0 (with the PG_GEOMETRY keyword) have then exhibited some bizarre rendering issues when rendered using GeoServer. Above, the Landscape Project data appears to have some random triangular polygons that stretch beyond New Jersey’s western border. We used ArcGIS to process this layer as I wanted a Union (ESRI not PostGIS terminology) between Landscape and NJ’s municipality layer. ArcGIS Desktop’s method of storing multipart polygons in PostGIS seems valid – the table renders just fine in ArcGIS Desktop and QGIS – but is a little different than what GeoServer’s rendering engine is expecting.

Here’s a “fixed” version that enables GeoServer to render the data correctly.

Proper way to render Landscape Project data.

The “fix” comes in the form of a simple table view. By forcing the geometry field through ST_Force_Collection and then setting the table view up as a new WMS layer, the problem polygons disappear. Frankly, I was surprised this worked, but I assumed that if ArcGIS and QGIS could handle the data, the problem wasn’t the data, but GeoServer’s ability to process it. ST_Force_Collection is just enough to massage the data into a GeoServer-friendly form.

I’m not sure if it’s something to do with ArcGIS’s PostGIS support directly, but I have already experienced issues with weirdness due to inconsistent handling of circular features between ArcGIS, PostGIS and OGR. I’m going to generate a shapefile of some bizarre multipart shapes and see how the shapes are handled via SDE & PG_GEOMETRY and well as OGR to PostGIS. I’ll share my findings here along with the source data.

Posted in Data, Technology, Visualization, Web Mapping | Tagged , , , , , | 1 Comment

Beach erosion in Ocean City

While not as hard hit as the towns near Seaside Heights, Ocean City felt the effects of Sandy and is still in the process of cleanup. Back in July, I took a Gigapan photo of the Ocean City beach from the south side of the Music Pier.


View on Gigapan.com

Two weeks after Sandy, I took another shot from roughly the same spot.


View on Gigapan.com

The dunes present in the top photo are completely gone. Many of the shops fared well, a little broken glass and best signs, but overall, intact.

Here’s another panorama of the boardwalk in front of the Music Pier.


View on Gigapan.com

Posted in In the News, Technology, Visualization | Tagged , , , | Comments Off on Beach erosion in Ocean City

From NOAA imagery, extent of Seaside Heights damage can be seen

The amusement piers in Seaside Heights have been significantly damaged.

Funtown Pier partially destroyed.

Funtown Pier. Click through for full size.

A roller coaster at Casino Pier can be seen sitting in the surf. Two buildings at the end of the Pier are entirely gone.

Casino Pier. Click through for full size image.

For those that are unfamiliar with the two piers, here’s a before and after.

Funtown Pier, before and after.

Casino Pier, before and after

Posted in Aerial Photography, In the News | Tagged , , , | Comments Off on From NOAA imagery, extent of Seaside Heights damage can be seen

Post Sandy imagery available from NOAA

NOAA‘s web viewer of post-Hurricane Sandy imagery is now live. Many portions of the Jersey Shore received incredible damage. Below are some before and after pictures compiled from the viewer. I’m hoping to get NJ’s March 2012 imagery available as a standard baseline for comparison. Click for full size images.

Brick Beach 3

Mantoloking Bridge

Lyman Street, now an inlet

Atlantic City Boardwalk, by the casinos

abandoned portion of the AC Boardwark, now destroyed

One bright note: Lucy the Elephant survived!

Posted in Aerial Photography, In the News | Tagged , , , , | Comments Off on Post Sandy imagery available from NOAA

Hurricane Sandy live data resources

As Hurricane Sandy is about to make landfall somewhere in NJ or DE, keep safe and dry. Move to a safe location and stay there. Don’t attempt to travel after dusk due to reduced visibility and the intensifying storm.

Once you’re somewhere safe and have power and Internet access, you can keep apprised of the storm through the following links. I’ll keep the list below here updated throughout the storm.

Live Maps & GIS Data

Utilities

Crowdsourced Reporting

Offices of Emergency Management

 Static Data and Maps

Posted in Data, Events, In the News, Technology | Comments Off on Hurricane Sandy live data resources

Pulling data from OSM using XAPI, bringing it to ArcGIS

I was recently asked by a friend if I knew of an ArcGIS-friendly data set representing the highway interchanges within New Jersey. I knew the information was already available in OpenStreetMap, but how could I quickly pull just the highway interchanges out of OSM and bring them into ArcGIS?

Using the XAPI, I was able to pull just the relevant information from OSM. In this case, I only wanted nodes that contained a highway key with the value of motorway_junction. I used MapQuest‘s XAPI service and quickly generated a URL using a nifty map mashup. I had to manipulate the XAPI URL manually to change the bounding box; New Jersey was too large to search with a statewide bbox, so I broke the state into four portions by changing the latitude ranges. Making smaller requests for the part of the state between latitude 39 and 40, then 40 and 41, allowed my request to go through without timing out.

I was then left with four XML files that contained the highway interchanges. ArcGIS doesn’t know what to do with this, so I whipped up some Python to convert all four tables into one tab-delimited file. This code (included below) can then be used to add the points to ArcMap using the Add XY Data functionality.

All said and done, this project took me about an hour. I was able to send the data off in about 30 minutes, but then spent another thirty minutes cleaning up the script, modifying it to spit out any available key, not just the “ref” tag. The script is GPL, free for you to use as you see fit. Just remember to give credit where it’s due, attributing your data back to OSM.

Taking the time to do this for a friend is a good reminder of why I push our students to do more than just the GIS curriculum. Being familiar with OSM gives you a different point of view on GIS data, schema, and community than what you would gain from most GIS courses and entry-level positions. GIS is already about problem solving; being able to do some basic programming will enable you to tackle much larger problems efficiently. And providing technical assistance to others just makes you a better person.

Posted in OpenStreetMap, Technology, Tools and Scripts | Tagged , , , , , | 1 Comment

New Jersey Bike Map public meeting

detail of the NJ Statewide Bicycle Map.

This Thursday, May 10th, I will be attending the final public meeting to discuss the draft New Jersey Statewide Bicycle Map and Resource Guide. I attended the first meeting at Cumberland County College, but will be again providing comments at this final meeting to be held at NJ DOT Headquarters. I’m very much a single-use public commenter, as I really do not have many concerns about the map at all; frankly, it’s beautiful and I plan on getting copies as they are data-rich and rendered by an excellent set of cartographers. My concern is the data-richness – this map is detailed, and for DOT or the consultants to not release the underlying data along with the maps would be a sin.

Let me take a step back and explain how I first found out about this project. Dave Heller from SJTPO posted an email to the New Jersey Geospatial Forum discussion list, asking the group if there was any bicycle facility data that was publicly available and more recent than 2007.

Hello:

We have a bicycle/pedestrian GIS layer from 2007.  I wanted to find out if people here are aware of anything more recent.  Specifically, we are interested in South Jersey.  Please let me know if you are.

Thanks.

David Heller

Being the great resource that it is, within 10 minutes another member of the NJGF mailing list, Robert Blash, pointed Dave to the Statewide Bicycle Map project web page.

Take a look at http://bikemap.com/njbike/

NJDOT has a project to map out biking areas in NJ.

Robert Blash

I was in the middle of writing an email to Dave & the list about using OSM data for bicycle facilities when Robert’s email came in, so I took a look at the site. While I was glad that DOT is looking to create a statewide cycling map, I was a little dismayed by what I saw on the site. My email to the list follows:

CloudMade has OSM-sourced data available as ESRI Shapefiles.
http://downloads.cloudmade.com/americas/northern_america/united_states/new_jersey#downloads_breadcrumbs

You can see what bike/ped facilities are in place using OpenCycleMap: http://opencyclemap.org/
If you take a look at Glassboro – http://opencyclemap.org/?zoom=15&lat=39.70289&lon=-75.10368&layers=B00 – You will see the Glassboro-Monroe bikeway and the pedestrian connections throughout campus. Some of our bike racks are on OSM, but not all of them.

Regarding the bikemap.com site, I contacted DOT a few years ago regarding the old Bike Maps that were released, looking for the underlying data. The maps were produced by the RBA Group. DOT informed me that the PDFs were the only deliverable and that DOT did not have ANY geospatial data related to bicycle lanes, routes and facilities. bikemap.com is run by RBA Group. Will this project be no different and community-sourced bike data be locked up by a private consulting firm?

You can register for the site and leave comments. Please take a few minutes and let them know that the GIS data behind the map should be made publicly available. Comment #2 is what deliverables are important; click “electronic file”. For question #4, comments, I left the following:

“Please make sure that all data collected through bikemap.com relevant to the production of the cycling routes and facilities is made available to the citizens of New Jersey by placing the data online under an open data license or released as public domain.”

If this project is funded by both NJ DOT and US DOT, the results of it should be open and accessible.

Peace, love, and open source,
John

Merrilee Torres, from Burlington County, replied to the list mentioning that Steve Spindler Cartography (who owns bikemap.com) is involved in the project and that echoing my concern for public release of the data, as the project managers, RBA Group, solicited the counties for bicycle facilities data.

My understanding is that Steve Spindler Cartography is doing the mapping for the state bike map but yes, RBA is managing it.  I do know that they were trying to collect as much source data from Counties as possible which gives even more credence to the idea that the data should be publicly available.

Merrilee J. Torres, GISP

I then replied to Merrilee and the list, expanding on my position on government and open data:

I don’t want to suggest that the State (if it’s even legally allowed to do so) adopt some sort of license like the GPLv3 where derivative works are required to be released under the same terms; open and redistributable. If that were the case, it could be argued that anything sourced off of aerial photos would then need to be released. There’s a massive amount of value in having the State’s data open, however it does seem like the public is getting short shafted when public data gets vacuumed up and then locked away. There is some value-added in performing the work that merits compensation to the consultant, but if the State is the one footing the bill, it should be stipulated that the product is owned by and distributable under whatever terms the purchaser/the State deems fit.

The paper NJ Bike maps are going to be great to have. Let’s face it, while many GIS geeks seem to also be biking/outdoor enthusiasts, the reverse is far from true. The public is going to need the paper maps. If the data is released, there’s great opportunities for the GIS geeks to perform some analysis that either improves the data further or could help influence policy relating to bicycle use in the state. Open government data allows many great ideas and projects to come to fruition; producing data, research and applications far beyond the scope of the original project.

And if the data was already there and freely accessible, Dave Heller would have found it and y’all wouldn’t have been subjected to my open data rants! 🙂

John Reiser

The conversation then took a couple turns back on track, to locate some additional information for Dave, then died out. Then a month or so later, I received notice of the public meetings. On April 24th, I rode my bike to work, checked out a ZipCar and went down to the CCC campus for the public meeting.

The meeting was conducted by two staff members from RBA and a staff person from Baker. After explaining the process and the methodology used to rate the state’s roadways for bicycle suitability, they opened it up to comments and questions. I asked about the intent of the map; “Who is your audience?” was met with a “Yes,” followed with an explanation that they intend the map to be useful to anyone.

I voiced my concern with such a sentiment. Paper maps are not going to serve everyone. While most people cycling for recreation are almost guaranteed to benefit from the map, it’s still in paper form. There’s a cost to print and deliver. And even if you provide PDFs, there’s few opportunities for individuals to print large-format maps without a visit to a printer.

Cycling is a key part of commuting, whether it is the sole mode or used to make connections, like above. About 50 bikes were locked outside of Harrison's Path station on a rainy weekday.

Beyond that, what about those that bike because it is their only available mode of transportation? Are the maps and guidebooks going to be available to them? Are the books only going to be available in English? Take a visit to nearly any suburban chain restaurant and you will see a few bikes locked up in the rear, surrounded by a sea of automobile parking. Those bikes don’t belong to a suburban family of four, they belong to those cooking the meals and washing dishes.

If the data behind the maps are made available, the utility of this endeavor increases exponentially. Paper maps serve a purpose and are nice things to have, but in terms of providing for cyclists, planners, social workers, public advocates, and geeks like me are not going to be able build upon a PDF map. Providing the data in a free (libre, not gratis) form enables so many ancillary players to tap into the resource provided and expand its benefits to a larger audience.

Jersey looks devoid of regional connections on OpenCycleMap. I've personally added some smaller tours and bike paths, but there's much more left to add.

I’m attending the meeting at DOT mainly due to the response I received from the individual from Baker regarding the previous set of bicycle maps. When asked about why the data behind the previous set of maps was not made available, the answer I received was that the maps “were done in Illustrator.” Yes, the set of maps (available to download at no-cost from the link above) were likely modified and aesthetically improved in Illustrator. But only Illustrator? Considering the state has GIS data for the millions of miles of roadway in the state, why was only Illustrator used? Fine, forget the old maps for now – the new maps have the tours available on them, along with all of the roadway condition information. There is no possible way that all of that information was managed effectively outside of GIS. And if it was an entirely Illustrator produced product, why is the State paying consultants to be inefficient?

I was told that the data behind the new maps is definitely GIS data, but that the staff of DOT will ultimately decide if the data will be released. First, of all, I intend to submit an OPRA request for the data if it is not released when the project is deemed complete and the paper maps & PDFs are released. Secondly, why wouldn’t DOT want the data released? They already provide GIS data and maps in multiple forms for any use.

I spoke with Elizabeth Cox from RBA after the meeting and she encouraged me to provide written feedback. I will likely print this blog post (how quaint) and will submit it at the public meeting. I may decide to write something more formal for submission and will update this post to include it if written. At the least, I intend to attend the meeting and directly ask the individuals at DOT what are the future plans for this data. I also encourage you to provide feedback via their contact form. I’ll provide updates after the meeting Thursday.

Posted in Cycling, Data, Government, OpenStreetMap, OPRA, Planning, Technology, Transportation | Comments Off on New Jersey Bike Map public meeting

Announcing LearnWebMapping.com

I am pleased to announce that I have started another blog, LearnWebMapping.com. This summer, I am offering a Web Mapping & GIS Services course through Rowan University Online. To promote the course and to help keep myself mentally focused on web mapping concepts and technologies, I set up Learn Web Mapping. I will post as often as possible – aiming for daily – with tips, announcements, and articles related to web mapping. At this time, I intend to keep the blog ad-free; actually, the only advertisement being for my online course.

Go check it out. The first article is on the Digital Blue Ridge Parkway project from the University of North Carolina. I give an overview of the map itself, and then some details on the technology used to the make the web map.

I welcome any feedback you have, either here in the comments or via Twitter.

Posted in Updates, Web Mapping | Tagged , , , | Comments Off on Announcing LearnWebMapping.com

Zipcar is here!

Chelsey and Mike approve of Zipcar

Chelsey and Mike approve of Zipcar

The Rowan University Facebook account recently posted some not-so-clear messages announcing the arrival of Zipcar on campus. I was thrilled to hear the news! Having a cheap rental car nearby has prompted me to reconsider owning two vehicles. I live just over a mile from Rowan and often ride my bike or walk to work. Why should I pay for a car I don’t use? Continue reading

Posted in Cars, Transportation | Tagged , , | Comments Off on Zipcar is here!

Finals season is upon us.

Fall Final Exam week is now here and I have been dealing with an unending stream of last-minute, panicked questions about the software and the assignments. This semester, I have two sections of GIS I which is required for a few majors. Until our students make it through GIS II (not required except for GIS majors, so usually only the Kool-aid drinkers get that far), they have a love-hate relationship with ESRI’s flagship product. At the start of each semester, until they have had a significant amount of experience with the software, students react with groans as each new assignment is given.

"What's in the box?" "ArcGIS 10 for Desktop Standard."

After a long 15 weeks, students will gain the experience and confidence needed to tackle some difficult, real-world problems with GIS.

"Mastering GIS is akin to riding your first sandworm."

Continue reading

Posted in Uncategorized | 1 Comment