Replicate data from MongoDB to Autonomous JSON Database

Learn to replicate data from MongoDB to Autonomous JSON Database using OCI GoldenGate.

Oracle Autonomous JSON Database is a cloud document database service that makes it simple to develop JSON-centric applications. It features NoSQL-style document APIs (Oracle SODA and Oracle Database API for MongoDB), serverless scaling, high performance ACID transactions, comprehensive security, and low pay-per-use pricing. Learn more about Autonomous JSON Database.

The OCI GoldenGate Big Data deployment type supports no down-time migrations from MongoDB to Autonomous JSON Database. OCI GoldenGate supports both Initial Load Extract and Change Data Capture (CDC) extract from MongoDB.

This quickstart details the process to configure OCI GoldenGate for no down-time migrations from MongoDB to Autonomous JSON Database.

Before you begin

To successfully complete this quicktart, you must have the following:

  • MongoDB replica set configured.
    • OCI GoldenGate Big Data capture uses operations log (oplog) to read the CDC records. The oplog is a capped collection that keeps a rolling record of all operations that modify the data stored in your databases. Oplog files are created in MongoDB when Replicat set is enabled. MongoDB Atlas comes with a preconfigured Replicat set configuration. For on premises MongoDB, you need to deploy a replica set.
  • MongoDB 3.6 & later.
  • OCI GoldenGate support for capture of following operations: INSERT, UPDATE, DELETE.

To learn more, see Using Oracle GoldenGate Capture for MongoDB.

Task 1: Create the OCI GoldenGate resources

This quickstart example requires deployments and connections for both the source and target.
  1. Create an Oracle Autonomous JSON Database.
  2. Create a Big Data deployment for the source MongoDB and target Oracle Autonomous JSON Database.
  3. Create a MongoDB connection.
  4. Create connection for target Create an Autononous JSON Database connection.
  5. (Optional) If your Big Data deployment doesn't have a public endpoint, then create a connection to GoldenGate, and then assign this connection to the Big Data deployment.
  6. Assign MongoDB and Autonomous JSON connections to the Big Data deployment.

Task 2: Create and run the Change Data Capture Extract for MongoDB

  1. In the OCI GoldenGate deployment console, ensure that you're on the Administration Service Overview page, and then click Add Extract (plus icon).
  2. For Source, select MongoDB from the dropdown and select the Extract Type as Change Data Capture Extract.
  3. On the Extract Options page, complete the following fields, and then click Next:
    1. For Process name, enter MCDC.
    2. For Connection Alias, select the name of your MongoDB connection from the dropdown.
    3. For Trail Name, enter M1.
  4. On the Parameter File page, ensure the source mapping includes TABLE source.*;
  5. Click Create and Run.

Task 3: Create and run the Initial Load Extract for MongoDB

  1. In the OCI GoldenGate deployment console, ensure that you're on the Administration Service Overview page, and then click Add Extract (plus icon).
  2. For Source, select MongoDB from the dropdown and then select Initial Load Extract for Extract type.
  3. On the Extract Options page, complete the following and then click Next:
    1. For Process Name, enter MIL.
    2. For Connection Alias, select the name of your MongoDB connection from the dropdown.
    3. For Trail Name, enter I1.
  4. On the Parameter File page, make the following changes:
    1. Locate the EXTTRAIL line and replace $extfilePath with your trail name. For example:
      EXTFILE I1
    2. Define the source mapping as TABLE source.*;.
      Note

      This is the source database/collection mapping. TABLE *.* results in extracting from all the databases/collections.
  5. Click Create and Run.

    When MongoDB Initial Load Extract runs successfully, you will see the statistics in the extract report file.

Task 4: Create and run the Initial Load Replicat

  1. In the OCI GoldenGate deployment console, ensure that you're on the Administration Service Overview page, and then click Add Replicat (plus icon).
  2. Add the Initial Load Replicat.
  3. On the Add Replicat page, under Replicat type, select either Classic or Coordinated, and then click Next.
  4. On the Replicat Options page, complete the following form fields, and then click Next:
    1. For Process Name, enter a name, such as RIL.
    2. For Trail Name, enter the name of the Trail from Task 3 (I1).
    3. For Target, select the target ORACLE_AUTONOMOUS_JSON_DATABASE from the dropdown.
    4. For Available Aliases, select Autonomous JSON connection from the dropdown.
  5. On the Replicat Parameters page, you can specify parameters to further configure your Replicat, and then click Next:
    MAP *.*, TARGET *.*;
  6. On the Properties page, review the properties, and then click Create and Run.
You return to the Overview page, where you can review the Replicat details. When replicat starts successfully, you'll see it in a running state with a green check. You can review the replicat details and statistics to confirm the replication.

Task 5: Create and run the Change Data Capture Replicat

  1. In the OCI GoldenGate deployment console, ensure that you're on the Administration Service Overview page, and then click Add Replicat (plus icon).
  2. On the Add Replicat page, under Replicat type, select either Classic or Coordinated, and then click Next.
  3. On the Replicat Options page, complete the following form fields, and then click Next:
    1. For Process Name, enter a name, such as MCDC.
    2. For Trail Name, enter the name of the Trail from Task 2 (M1).
    3. For Target, select MongoDB from the dropdown.
    4. For Available Aliases, select assigned MongoDB connection from the dropdown.
  4. On the Parameter File page, you specify parameters to further configure your Replicat, and then click Next:
    HANDLECOLLISIONS
    MAP *.*, TARGET *.*;
    Note

    Add HANDLECOLLISIONS to resolve the issues with duplicate or missing records while applying the replicat.

    HANDLECOLLISIONS parameter has a negative impact on the performance and that’s why it is recommended to use as needed. When your source & target is synced, you can stop the CDC replicat, remove HANDLECOLLISIONS and re-start the replicat.

  5. On the Properties page, review the properties, and then click Create and Run.
You return to the Overview page, where you can review the Replicat details. When Replicat starts successfully, you'll see it in a running state with a green check. You can check the Replicat details and statistics confirm the replication.