Transcription
Enhancing spatial webserviceswith analitics – case LNEDATABANKDATABANK ONDERGROND VLAANDERENDatabase of the soil and subsoil of FlandersONDERGRONDMercatorNet cooperationVLAANDERENPlace du DOV dans l'infrastructured'information géographique en Région flamande,accessible via Geopunt.GWF, 25/05/2016Marleen Van DammeDOV- and MercatorNet coordinator06/11/2014 - Marleen Van Damme - DOV-coördinator
OverviewSituation of the case LNEOpen data in FlandersWhat is DOV?What is MercatorNet ?Enhancing spatial web services of LNE with analiticsSesamKibanaZabbixGoogle monitorSpatineo MonitorConclusions
Open data in Flanders versus SDIMercatorNetDOVGeopunt
What is DOV?URL: http://dov.vlaanderen.beDOV is a cooperation in Flanders:First established in 1996Open data online since 20023 entities are the core partners:Mobility and Public Works Department (MOW)Environment, Nature and Energy Department (LNE)Flemish Environment Agency (VMM)
Which data in DOV?DOV offers data and information on thematic areas related tosoil and subsoil:geotechnicsgroundwatersoilgeologyAll data is available as opendataReuse is allowed“gratis open data licentie Vlaanderen V1.2”
What is MercatorNetCooperation within the government of FlandersMobility and Public Works (BD MOW)Environment, Nature and Energy (BD LNE)Physical Planning (BD RWO)Common infrastructure to support all GIS users of thepartners involvedArcSDE Database to be replaced by new Read Onlydatabases (PostGis)Common infrastructure to support publication of geodatavia servicesAll data is available as opendataReuse is allowed“gratis open data licentie Vlaanderen V1.2”
Architecture for DOV/MercatorNetBoth DOV and MercatorNetPostgreSQL, PostGISServicesCatalog (Geonetwork): CSWDOV: 1000 metadata filesMercatorNet: 980 filesOGC-services (Geoserver) :WMS, WFS/WCS, WMTS –DOV: 470 layersMercatorNet: 130 layersCommon application forpublication: called“Mercurius” for DOV“Ariadne” for MercatorNetDOV only:Applications fordatamanagement andviewingJava, GWTGeotools, GeomajasVerkenner
Mercurius / Ariadne
Analitics – several tools in useSesam since 2011 Logging of all requests onGeonetwork and geoserverOwn application developed in 2011 –still runningMonitoring Website’s Uptimewith Google Docs Real time uptime check, requestevery 5 minSpatineo since 2015 Real time monitoring of responsetimes, request every 5 minUsage analitics : based on ownlogfiles, shipped to SpatineoKibana LoggingReal time monitoringZabbix monitoring
Usage of DOV –Sesam compared to 4051.531Furtherinvestigation neededto understand thedifferences
Spatineo Monitor –meters and alert settings
Spatineo – warnings and alerts
All warnings combined in onemailbox – who’s the first ?
Zabbix versus Spatineo
What happend yesterday ?DOV and MercatorNet weredown for 30 à 50 minutesProbably caused by wrong changes of configurationrelated to DNS on the shared infrastructure .
Spatineo - Performance testingDOVVector complex legendCap. 470 layersWFS – GML2MercatorNetrasterVector light legendVector light legendWFS –app/gml xmlTestscenarioDOV: 50 req/s in totalMercatorNet: 56 req/s intotalFirst test on 19/04/2016Initial set up knowingexistence of performanceissuesSecond test on 24/05/2016MercatorNet – no changes –Geoserver running on 2nodesDOV – Geoserver running on4 nodes in stead of 2
Spatineo - Performance testing –results on 19/04/2016DOVMercatorNet
Spatineo - Performance testing results on 24/05/2016DOVMercatorNet
Kibana during the test comparedto regular usageDOV - Requestsfrom spatineomonitoring andperformance testincudedDOV – requestsfrom “real” usersduring thetesting timewindow
Kibana versus SpatineoSpatineo monitor incuded10.000 requests from Spatineo during the visible time frameCorresponding views in Spatineo for the same daySpatineo monitor not incudedMax: 3500 requests per min almost 60 requests per secondKibana: all req on geoserver DOVSpatineo: Only req. on WMS DOV
Reporting with Spatineo
ConclusionsProviding open data is challengingUsing the same services that had been set up forpublication of data in own applications leads to even moreneed to have insight in usage and performanceSeveral tools from different points of view have to be inplacePositive feedback from users is motivatingHaving the possibility to use DOV, our users “enjoy”paying taxesDue to open data policy in Europe – governemetal tasksrelated to it have become vital for the whole society. Costsare worth spending due to gain for all involved target users.
QUESTIONS ?Contact us at dov@vlaanderen.be
Monitoring Website's Uptime with Google Docs Real time uptime check, request every 5 min Spatineo since 2015 Real time monitoring of response times, request every 5 min Usage analitics : based on own logfiles, shipped to Spatineo Kibana Logging Real time monitoring Zabbix monitoring
