Field skipping & field renaming from RDBMS

Field skipping and field renaming are an advanced feature of GlueSync. All of the operations involved in that features are all performed on-the-fly without involving local caches or any form of persistence inside GlueSync, this makes the overall process faster, secure and consistent.

A step-by-step video tutorial on how to configure could be found at this link in the Tutorials and use cases section of the docs.

On-the-fly field skipping

Leaving the default configuration file as documented in the “Basic” section, that means leaving just the table name, will let the replicator transfer and convert all the columns present in the specified table. There are certain scenarios in which sometimes you would like to not sync certain columns from your SQL tables. GlueSync supports field skipping, declaring them directly inside the configuration file will tell the engine to skip the specified column name from the parent table. Here below you can find an example of the sourceEntities key declaration to support field skipping:

"sourceEntities": {
    "TABLE1": {
        "mapping": {
            "COLUMN1",
            "COLUMN2",
            "COLUMN3"
        }
    },
    "TABLE2": {...}
}

Notes and considerations

GlueSync will not transfer and sync any other column present in the specified table other than the ones typed inside the table fields definition. You can add as many new fields your table has to put them in sync with NoSQL, just add them in the definition file and be sure you restart the service / container to reload the updated configuration file.

Your JSON document inside the NoSQL target database won’t update automatically as you add new columns in the config file: they will receive the new field as soon as its row gets updated in the SQL table. In case you need to have the newest fields available you may want to trigger that with an update query statement against the dataset in the specific table.

On-the-fly field renaming

It is likely common to find huge column names inside SQL tables and to avoid wasting space in the target NoSQL database GlueSync supports what is called on-the-fly field renaming.

To enable that feature per column name you can follow this example below.

"sourceEntities": {
    "TABLE1": {
        "mapping": {
            "HUGECOLUMNNAME1": "NEW_COLUMN1",
            "VERYLARGECOLUMNNAME2": "NEW_COLUMN2",
            "LARGESTCOLUMNNAMEEVERSEENINASQLTABLE3": "NEW_COLUMN3"
        }
    },
    "TABLE2": {...}
}

Notes and considerations

Enabling on-the-fly field renaming will automatically enable field skipping, so for columns not present inside the table definition the content won’t be replicated on the target NoSQL database. Field skipping and on-the-fly field renaming can be used simultaneously. GlueSync will of course translate the keys coming from the NoSQL database back to its original value [be sure they are configured in the proper section in the NoSQL to SQL module].