Database migration and modernization services for an Oil and Gas company

Contact us

Services

Database Migration
Database modernization
High scalability
DB optimization

Executive Summary

The client is a leading name in Oil and Gas industry. Facing multiple scalability issues, they engaged Simform for a database migration strategy and build a future-proof-scalable database. This database is currently being consumed by an Oil and Gas geolocation software to identify potential locations for Upstream operations.

Challenges

PostGis to MongoDB migration

We had to migrate millions of rows of data from PostGis to MongoDB.

Unsatisfactory data records

Some records in the data were not satisfactory to build an efficient database. Also, some tables were making it almost impossible to merge the data.

Solution

Generating DB reference structures

MongoDB doesn’t creates the same indexing as PostGis upon migration. We built reference structures to facilitate migration from RDBMS to NoSQL. Without building reference structures, direct migration of the database would create a corrupted database.

From PostGis to GeoJSON

We converted shapefiles into PostGis using qGIS. PostGis stores geographical information into binary, and decrypting for MongoDB is difficult. We then converted this geographical information into plain text, finally, converting it into GeoJSON.

Improving records quality

When we looked into various shape files, we found extremely high granularity and multiple duplications in the existing data. To import an error free database, we optimized polygon points in shapefiles to remove any duplication within these records.

Solution

Simple data model

We created a simple yet very integrable, high maturity data model. This data model further facilitated building business applications under very restricted resource environments. The solution leverages Google maps with on demand data binding.

Merging multiple tables

Most data tables came from different sources, many had different formats, granularity and indexing. We unified and merged these tables into a single document to create a flexible schema design for MongoDB.

Sharding and Batch migration

The data was migrated in batches, each batch was evaluated for errors upon successful migration. As the database scales, we wanted to make sure that complexities and growth limitations remain simple. With elastic deployments, we improved business application performance during load spikes.

Business Impact

Before moving to our modern database, client was facing issues while scaling with millions of rows of data. After migration, their business can now consume and process 20 Million+ rows of data on a scale, helping them leverage simplified and high performing data models to power enterprise applications and BI dashboards.

Speak With Our Expert