Big Data Analytics in the Geo-Spatial Domain

Empowering geo-spatial analytics with database technology

Image: NASA Earth Observatory (CC License)

Project Leader: Prof. Martin Kersten (CWI) eScience Research Engineer: Dr. Romulo Pereira Gonçalves Co-applicants: Prof. Henk Scholten (VU University Amsterdam), Dr. Sisi Zlatanova (Delft University of Technology) & Dr. Milena Ivanova (Netherlands eScience Center)

Digital 3D city models play a crucial role in research of urban phenomena; they form the basis of flow simulations (wind streams, water runoff and heat island effects), urban planning and analysis of underground formations. Urban scenes consist of large collections of complex objects which have rich semantic properties, such as materials and colors. Modeling and storing these properties indicating the relationships between them is best handled in a relational database.

Database management systems (DBMSs) are a well-established solution when it comes to archiving, filtering, analysis, and correlation of large data collections. Ability to perform analysis near data is one of the key requirements identified by the 4th Paradigm to handle the data deluge. A single spatial DBMS offers functionality for geo-spatial modeling and management of semantic properties in one place, thus avoiding the need for multiple software tools associated with high volume data transfer and format transformations.

The provision of spatial and geo-spatial features in database systems needs to be extended and brought to maturity to fulfill the requirements of real-world scientific applications. A class of DBMSs, called column-stores, has proven efficiency for analytical applications on extremely large datasets. In fact, all major DBMS vendors have extended their product spectrum with column-oriented solutions to address the needs of analytical applications. The aim of this project is to develop and mature the spatial features of the column-store open-source MonetDB. It has established a track record in high-performance analytical applications and demonstrated its ability to inject database technology successfully in several science domains, such as astronomy, remote sensing, seismology, and navigation.

The technology will be applied to a concrete use case of the Port of Rotterdam in which a 3D GIS is built to aid various multi-stakeholder construction projects where new structures are built in, on top of and around the existing port (underground) infrastructure. Extending and modifying the port is challenging as it is home to many different companies that often cover extensive areas and manage vast (underground) infrastructures such roads, pipes and cables. The port thus requires a 3D GIS that is able to store all harbor assets and analyze existing assets with future interventions and detect conflicts. The 3D GIS currently being built is aimed at collecting data from different sources and formats (BIM and GIS) and converting it to a common format to enable 3D operations and analyses such as 3D intersections, 3D buffers as well as simplification and generalization of GIS and (especially) BIM models for visualization purposes.

The expected project outcome is as follows:

The development will rely as much as possible on existing open-source tools and libraries. The proposed (geo-) spatial data analytics tools will extend the eScience Technology Platform (eSTeP) and be offered as an associated technology available to the NLeSC projects and other eScience projects in national and international context.

Participating organisations

CWI
Environment & Sustainability
Environment & Sustainability
Natural Sciences & Engineering
Natural Sciences & Engineering
Delft University of Technology
Netherlands eScience Center
Vrije Universiteit Amsterdam
MonetDB Solutions

Impact

Output

Team

MK
Martin Kersten
Principal investigator
CWI
MI
Milena Ivanova
eScience Engineer
Netherlands eScience Center
Jason Maassen
eScience Engineer
Netherlands eScience Center
Romulo Gonçalves
Romulo Gonçalves
Technical coordinator
Netherlands eScience Center
Oscar Martinez Rubi
Oscar Martinez Rubi
PN
Pirouz Nourian
Co-Applicant
Technische Universiteit Delft
KAO
Ken Arroyo Ohori
Co-applicant
Technische Universiteit Delft
HS
Henk Scholten
Principle Investigator
VU Amsterdam
SZ
Sisi Zlatanova

Related projects

nD-PointCloud

continuous level representation for spatio-temporal phenomena in Open Point Cloud Maps

Updated 7 months ago
In progress

Visual Storytelling of Big Imaging Data

Storytelling as a means of visual data communication

Updated 13 months ago
Finished

Algorithmic Geo-visualization

From theory to practice

Updated 15 months ago
Finished

Error Detection and Error Localization

Approaches for radio telescope system health management

Updated 5 months ago
Finished

3D Geospatial Data Exploration for Modern Risk Management Systems

The country below sea level

Updated 11 months ago
Finished

Improving Open-Source Photogrammetric Workflows for Processing Big Datasets

Processing large datasets on consumer-grade computers

Updated 11 months ago
Finished

RT SAR

An architecture for real Time big data processing for AMBER

Updated 11 months ago
Finished

Massive Point Clouds for eSciences

Using point clouds to their full potential

Updated 11 months ago
Finished

Related software

Massive PotreeConverter

MA

Use parallel processing to quickly convert large point cloud data sets to the format used by the Potree viewer.

Updated 20 months ago
1 3

PattyAnalytics

PA

Library for aligning and scaling one point cloud to an other.

Updated 20 months ago
4