Configure Incremental Data Mart Load

This page will provide an overview of things to consider when creating Access Views for Information Sources supporting incremental load as the Data Mart source. The configuration details refer to the Aurena client.

In most cases the Access Views to be used by Analysis Models are based on On Line access. It is however recommended to use Data Mart (DM) access views as the base for creating Access Views for cases where the Data Mart access is based on Incremental Load. The reason being that for Incremental Load the DM table can be updated on a rather frequent basis, based on source table changes and only adding or updating as few rows as possible. The DM table can then be kept up-to-date by e.g. refreshing every hour.

This page only supplies some general advice. For more information, please refer to Data Mart administration and configuration in Aurena

Contents

Enable Data Access Type

When Data Mart access is chosen for an Information Source, it is advised to make sure that the Enabled Data Access Type is defined correctly.

This is done in the Information Source page

If Enabled Data Access Type is set to All it means that the entity (e.g. Fact) supports both On Line and Data Mart access.

Activating an Entity

Before starting a transfer to the Data Warehouse in SQL Server it is necessary to create the first snapshot in the Data Mart source table. Use the Entity Info - Incremental Load page.

There are two case to consider:

  1. If the Information Source is not Active and the Refresh Status is UNUSABLE, then the first refresh is handled by performing Activate


    1. Use the Activate command that will start an assistant (Activate Data Mart Sources)
    2. Select the Information Sources to be activated
    3. Skip Related Sources and Gather Statistics
    4. Verify and Finish
  2. If the Information Source is Active and the Refresh Status is UNUSABLE, then the first refresh is handled by performing Refresh

    1. Use the Refresh command that will start an assistant (Refresh Data Mart Sources)
    2. Select the Information Sources to be activated
    3. Skip Related Sources and Gather Statistics
    4. Verify and Finish

 

Even if the Refresh Mode is Incremental there will be a full refresh since the Refresh Status is UNUSABLE.

Note: It is possible to perform the refresh using filter conditions to reduce the number of transactions. Normally these types of conditions are handled on the SQL Server side.

When the first activate/refresh has successfully finished, the entity is now prepared for incremental update,

Creating a Refresh Category

The next step would be to make sure that entities supporting incremental loaded are refreshed on a regular basis.

To do this, start by creating a Data Mart Refresh Category and then, to this created category, add the entities to be refreshed with the same refresh schedule.

 

Setting up a Refresh Schedule

When the entities to be refreshed have been added to a Data Mart Refresh Category, setting up a refresh schedule for the category will be the next logical step.

Use either the Data Mart Refresh Categories page or the Data Mart Refresh Categories Details page and click on Schedule Refresh to navigate to the Database Task Schedule page. Configure the schedule and save.

Assume that a schedule is set up to execute twice a day, this means that the data mart sources connected to the Data Mart Refresh Category will be refreshed two times per day. The refresh is done according to the Refresh Mode in the Entity Info - Incremental Load page.