First, I want to apologize, I just got Internet access while at FOSS4G, so I couldn’t provide good coverage so far… But don’t worry, I’ll make sure you get all the most important announcements and news, probably some time next week. At least for the elements other Slashgeo editors won’t have already shared.
Meanwhile, here’s my notes and the results of the famous WMS Shoutout, which just ended minutes ago.
– WMS benchmarking, 8 teams this year instead of the usual 2.
– This year with Cardcorp, GeognoSIS, Constellation-SDI, ERDAS APOLLO, GeoServer, – Mapnik, MapServer, Oracle MapViewer, QGIS MapServer.
– Both open source and proprietary solutions compared with the same datasets, the same WMS requests, same output formats, etc.
– Testing 18 gigs of vector data.
– Testing 120 gigs of raster data.
– Using real Spain data.
– From 1 to 64 parallel clients requests (2152 requests total).
– Main winners are the users, because we are the ones who will benefit from the improvements done to the WMS servers (one rule of the Shootout is that modifications must become available to the users, I guess this also applies to proprietary software competitors).
– This year, benchmark includes bottlenecks from CPU, disk access, network access and remote database access.
– If you wonder, these guys really went deep in the benchmarking analysis (such as OS-level caching), take a look at the slides. They do try to do a serious benchmarking comparison.
– There was serious differences in servers which were disk-bound vs disk-unbounded (CPU-bound).
– Tests included dbf files of over 2 gigs.
– Of course, there is performance in terms of time to answer requests, but there is also the quality of the rendered tiles, which is more subjective.
– All contestants agree that this was a lot of work to participate to the Shootout, but it was worth.
– GeoServer got many great improvements for the sake of the Shootout, same for MapServer.
– Mapnik: even built their own new WMS server named Paleoserver.
– Oracle MapViewer: found quite a few bugs and bottlenecks, and yes, they plan to participate next year.
– QGIS MapServer: integrating desktop GIS and server. Missing time to get consistent results.
– Constellation-SDI: it’s open source software (LGPL). First serious work in performance, many challenges found.
– Cadcorp GeognoSIS: Windows-only server. Added many improvements for the sake of the Shootout.
– ERDAS APOLLO: exciting benchmarking project. Made many improvements too. They’re refusing to publish results because CPU-bound results cannot be compared with Disk-bound results. Yeah, contestants can step aside from displaying results for any reason they see fit.
– And now the results…
– MapServer wins first test, Mapnik second
– second test, MapServer first, GeoServer second
– third test, Constellation, map server GeoServer all doing good
– fourth test, MapServer generally faster
– fifth test, still MapServer ahead
– on the last test, Mapnik doing good
– Overall, one can say that MapServer can be considered the faster of them all at the moment. Of course, considering that some servers were able to cache in RAM better than others.
– For next year, the whole WMS process will start much earlier.
– The show ended by the WMS-song.. Pretty funny.
– Obviously, the full results will be available soon on the OSGeo website (search Benchmarking on their wiki).