Skip to content

Data Sync

Organizations generate large volumes of data across multiple systems, making it essential to consolidate information for analytics, reporting, decision-making and AI integrations. The Data Sync solution provides governed connectivity between IFS data and your own data ecosystem.

This solution seamlessly transfers selected data from the IFS Cloud Oracle database to the tenant's dedicated data lake within the IFS.ai Platform and then to your organization’s own data lake environment (Amazon Simple Storage Service - AWS S3, Azure Data Lake Storage - ADLS Gen2, Google Cloud Storage - GCP-GCS).

It allows you to use your preferred BI/analytics tool (Power BI, Tableau, Qlik, etc.) and to maintain a unified view of enterprise data.

Note

At present, Data Sync is supported only for IFS Cloud Service/IFS Managed Customers. Support for Remote Customers will be addressed in the future. Further, this solution is only available to the customers from IFS Cloud 25R2 Service Update 10, and IFS Cloud 26R1 Service Update 4 onwards.

Sync Source Types

The Data Sync solution supports the following three synchronization sources:

  • Tables
  • Data Lake folders
  • Data Products

Note

The sync source types available to you depend on your tenant's entitlements. If a source type is not listed when you create a sync job, it indicates that it is not included in your tenant's entitlements. At present, Data Products are not available to select as a sync source. This will be supported in a future release.

  1. Tables

    This option allows you to select individual tables for synchronization. You can select:

    • Tables linked to a specific Data Product, or
    • Tables that you have enabled for Data Streaming in IFS Cloud.

    Best suited for: users who need selective data access or want to enable data streaming only for specific tables.

    Visit the Data Streaming Enabled Tables page in the IFS Cloud Technical Documentation to learn how to enable data streaming for tables.

  2. Data Lake Folders

    This option enables folder-level synchronization within the tenant-specific Data Lake in the IFS.ai Platform. You can select folders from the following predefined folders:

    • Gold
    • Silver
    • Bronze
    • External and Batch folders within the Bronze layer

    When a Data Lake Folder is selected as the source type, all files and documents within the selected folder are automatically synchronized. This includes content such as documents, parquet files, and other file-based data stored within that folder.

    Best suited for: File and document synchronization scenarios where data is already organized by processing layer (e.g., Bronze, Silver, Gold) or by functional domain, and where table-level synchronization is not required.

    Note

    Files and documents within the Data Lake are synchronized up to the 5th level of the folder structure in each tenant’s Data Lake.

  3. Data Products

    This option syncs all tables linked to the selected Data Product. Selecting a Data Product ensures that every table associated with it is automatically included in the sync. Although Data Products may contain other artifacts, only the tables associated with the Data Products will be included in the sync.

    Best suited for: users who want a complete, governed dataset without managing individual table selections.

Data Movement from Source to Destination

Data Sync transfers selected data in two stages. First, it transfers the data from the IFS Cloud Oracle database to your tenant's dedicated data lake in the IFS.ai Platform. From there, it delivers the data to your organization's own data lake.

Whichever sync source type you choose, the content that gets synced is one of two kinds: tables, or files and documents. Each kind reaches the IFS.ai Platform data lake in a different way.

Tables

A Data Streaming Service, an internal platform mechanism, handles table data. It identifies the tables to synchronize, captures any changes made to them, and transfers those changes from the IFS Cloud Oracle database to your tenant's data lake.

To make a table available for streaming, enable it from the Data Streaming Enabled Tables page in IFS Cloud. For details, see Data Streaming Enabled Tables in the IFS Cloud Technical Documentation.

Files and Documents

Files and documents are transferred by the Data Pump, an existing IFS Cloud mechanism, which is triggered by a Workload Job Definition.

To learn more, visit the following pages in the IFS Cloud Technical Documentation:

Once your data is available in the tenant's data lake on the IFS.ai Platform, you can sync it to your organization's own data lake. To do this, create and manage a sync job in Data Services Designer within DSP.

To learn about Data Services Designer and Data Sync job configuration, refer to the following pages:

Note

Sync frequency for any synchronization source type can be set from 30 minutes up to several hours in DSP. The initial data load may take longer, depending on the volume of data being loaded and network conditions.