GUI Based Migration Studio - "Lift & Shift"

A migration tool to migrate Open source Database (MongoDB & PostGIS) to Oracle Database

Introduction

“… If necessity is the mother of invention..  

… then laziness is the father of invention….”

In reality..

The Sloth and the challenges stimulates innovation, which enhances productivity by adhering to the principle that “ there must be a better way !! ”

I am continuing this article with the aforementioned quote.

Disclaimer:

This is a personal blog. Any views or opinions represented in this blog are personal and belongs solely to the blog owner. Above article is purely based on my thought ,, my third process and nowhere related to my current working organization. This blog does not compare any databases and does not talk advantages or disadvantages source and target databases. This blog is written just to show the capability of easy application development by considering use case of migration process using Oracle APEX and Python.

Now, we move on article..

Wouldn’t it be beneficial to build the application without incurring any additional costs for the developing the application utilizing open source code and free development tool ?

In Detail :

With continuation of migration methodologies series, we will look an another approach to migrating MongoDB Collections to an Oracle Autonomous Database or Non-Autonomous Database using a GUI-based migration tool called “Migration Studio” (Lift & Shift).

We all are aware that (refer to my previous migration articles) there are various methodologies available to migrate open source databases like MongoDB collections to Oracle JSONDB based on the source type, such as Flat file, Object Storage, On-Prem, or Atlas. Based on source type, we need to adopt appropriate method. This chaos can make us difficult to select the optimal method.

MongoDB Collections to Oracle JSON

Migration Matrix

Case 1/ From MongoDB collections from OnPrem server we can migrate to Oracle JSONDB ( ATP or Non ATP) either through oracle SODA API or Oracle GoldenGate. Other methods are not right approach.

Case 2/ Oracle SODA API can be used to migrate data from MongoDB Atlas to Oracle JSONDB ( ATP or Non ATP). Other methods will not help.

Case 3/ if collections are available as Flat file in Object Storage, using Oracle MongoDB API or using oracle dbms_cloud package we can migrate to Oracle Autonomous JSONDB. Other methods will not help.

Case 4/ If collections are available flat file in File system, we need to use Oracle Loader for Non Autonomous Database or Oracle MongoDB API for Oracle Autonomous Database to migrate collections to Oracle JSON. Other methods are not optimal to use.

From above matrix,

- Oracle MongoDB API and dbms_cloud package can be used only for Oracle ATP database 

- SODA API and Oracle GoldenGate can be used for both Oracle ATP and NonATP database

- Oracle Loader can be used only for Oracle non-autonomous databases

PostgreSQL (PostGIS) to Oracle DB

Similar to this, there are numerous approaches ( refer my previous blogs ) to migrating PostgreSQL (PostGIS) to Oracle Spatial, each of which presents its own unique set of obstacles.

We’ll walkthrough the challenges of each case.

Case 1/, if we have PG binary dump (without spatial data) as source, using python modules , we can migrate to Oracle Database. But extensive coding is required.

Case 2/, if we able to access PG data (without spatial data ) directly from PostgreSQL server, either by using Oracle GoldenGate or Oracle SqlDeveloper we can easily migrate. But as of now today, both ( OGG, SqlDev) not supports Spatial Data type .

Case 3/, If PG data has spatial data ( Raster Data ) , it is highly recommended to use GDAL utility to convert PG spatial to oracle Spatial. However, if any of column of source table has date or time , it leads to timezone format error and skip entire record that leads to inconsistent of records in Target Database

Case 4/, if Spatial files are available as flat files (Raster Data or Vector data ), it is easy to load via Oracle Spatial Studio and push to Oracle Database , however Spatial studio does not support Raster Data.

Case 5/, For LiDAR files, we need to rely ONLY on LAS utilities

In an effort to eliminate above complications and simplify the migration process, I developed a GUI-based migration utility known as “Migration Studio — Lift & Shift.” , using Oracle APEX ( no-cost feature of the Oracle Database ) and Open Source Python language.

About “Migration Studio — Lift & Shift”

• Front End GUI created using Oracle Native Low Code APEX v23.2 ( One of the most well-liked enterprise low-code application platforms is Oracle APEX, which lets you create highly feature-rich, scalable, and secure web and mobile apps that can be used on-premises or in the cloud. It’s true that, developers can quickly develop and deploy compelling apps) with ORDS v23.6 runs on Oracle DB 19.6

• The internal program kernel developed using Python v3.9 modules.
  

How it works:

- Python Source code runs in OCI Compute VM in public subnet- Python Code called as API from APEX page.

- Based on credentials, End User can proceed stage by stage - Migration Log is spooled and displayed in console.

- If any collections are NOT migrated or fail in the middle, it can be re-executed again.

Unique Value Proposition of this tool:

- Since ORDS runs in public subnet, this tool can be accessed from anywhere. - It is GUI based, and Easy to use by non-experts as well

- Credentials are not stored in any manner. It is transited internally via API call between source and target

- Tool will take care of migration methodology based on the options selected.

Watch this video for complete Demo.

Conclusion

To make life easier, with immense effect I built a Migration Tool using Oracle Low Code APEX and Python modules. While developing, i have learnt many new things and over came many challenges that helped my learning curve.

Thanks for your time to reading this article.

If you need more information about this tool, don’t hesitate to ping me via mail2crsathishkumar@gmail.com

Happy Learning and Happy Sharing.