This guide describes the Synchronization functionality of the Mobile Framework, and is intended to provide an overview of the Synchronization Process, looking at the Synchronization principles, the physical process of syncing mobile applications, and providing some basic troubleshooting steps for Synchronization issues.
A Synchronization requirement is either generated by a change in IFS Applications, or a change in a mobile application. These changes can take the form of the following:
An Insert – New Data
An Update – An amendment to existing data
A Delete – The removal of existing data
The changes may either be stored and synchronized at a pre-determined time, or they may be immediately synchronized, depending on the type of data in question. The Synchronization Rules determine how each entity used by a mobile application is handled in terms of sending data to the mobile application.
The mobile application will perform a Synchronization upon the following actions:
When opening the Sync Monitor on the client
When performing a save on the client
When doing an Initialization of the client
Upon receiving a push notification from IFS Applications
On a pre-set frequency defined in IFS Applications (in Application Parameters)
Device Activation – The first time a mobile application connects to IFS Applications after being installed on a device, this registers the User, Application and Device in Installed Apps/Devices and performs an Initialization.
Device Initialization - An initialization of the mobile application clears the local database on the device and any Failed Transactions for the User, Application and Device combination. The initialization will then synchronize down all data that the User has permission set access rights to, comprising all static data, basic data, and transactional data related to the mobile user. There are two types of initialization, an initialization and a force initialization. An Initialization will send any pending transactions from the mobile application to IFS Applications first. It will then delete the local database, and resynchronize all data the mobile user has access rights to. A Force Initialization (which only becomes available after selecting an initialization) will immediately delete the local database, including any pending transactions.
Note: An initialization may require certain data to be re-processed in the mobile application as the transactions from the mobile application may be deleted from the local database or by deleting the Failed Transactions.
Initialization Button – Windows Client
Force Initialization Button – Windows Client
Delta Level Change Handling – The mobile framework keeps track of data sent to each mobile application, using optimized Delta Handling routines to ensure only changes are sent to the client. This is based on the Oracle Database Objkey/Objversion, the relevant Entity Default Where statements and any permission set filters that are set.
Transaction Handling – Transaction handling is handled using a transaction replay model in client to server transactions, where all relevant user actions are captured in the out queue in the mobile application, and then once a synchronization is invoked the queued transactions are sent and replayed in the correct order in IFS Applications. The standard transaction methods called are Insert, Change & Delete methods. State Event methods are Metadata driven and specify available actions and state events as per requirements. Server to client transactions are handled via a combination of database triggers that track changes, and a combination of push queues, synchronization tasks, and background jobs that ensure the data sent to the out queue is for that user, app and device.
Pre and Post Blocks - As mobile applications are simplified versions of IFS Applications they can make use of pre-processing blocks and post-processing blocks to perform additional validations, set default data for attributes and do other database processing before and after the standard transaction processing.
Reliable Messaging – The transactions sent from the mobile application adhere to the AtMostOnce and InOrder delivery assurances as defined in the OASIS standard WS-ReliableMessaging version 1.2. Basically, message duplicates will be discarded and the framework guarantees that transactions are replayed once, one at a time (i.e. never in parallel) and in sequence for a given User, Application and Device combination. These are the basic requirements for maintaining the integrity of the transaction flow as performed by the end user, and are needed for any application that can function in offline mode.
When offline it is of paramount importance to maintain data integrity between the mobile application and IFS Applications. Reliable Messaging is a key component for achieving this. All messages are tagged with a Message ID (unique per User, Device and Application). When a message is received, a read receipt is written to the out queue. This read receipt must reach the sender to confirm delivery of the message. If the read receipt is not received by the sender the message will be re-sent. On receiving a message a subsequent time, and if a read receipt already exists the receiver will just return the read receipt without re-processing the message.
The data on IFS Applications is classified as either a Batch entity or a Push entity. These classifications are used to differentiate between static, operational, and transactional data.
Batch Entities – Static operational and basic data – changes infrequently
Push Entities – Editable data on devices and related entities, usually related to a Work Order.
The Synchronization process uses Batch and Push jobs to update the server and the mobile device.
The Push entities are always processed as Push and Batch entities, and are therefore a combination of both processes. The Push process will handle data changes directly on the entity, the Batch process will handle indirect data changes on the entity such as giving Site access to the User and thus exposing more records to the User.
Each Batch entity has its own frequency that can be configured per installation. When that frequency expires, the entity will then be processed for each Mobile User, Device, Application and Entity. A comparison is done between the data already sent to the Mobile Application and the data in IFS Applications. If changes are found, then these changes are added to the Out-Message table per Mobile User, Device, Application and Entity.
For most batch entities, the default schedule is every 24 hours, during the night to minimize the impact on the synchronization process during the working day. The default schedule can be changed in Synchronization Rules, but careful thought should be given to this, as any additional Synchronization load may affect system response times, particularly in terms of mobile data transfer.
Push entities are primarily transactional/fast changing data entities and the push process is triggered on New, Changed and Removed records. On pushing a record the process can also consider all child related records. This is based on the model/design of the push entity. For example, pushing a WO will include all Materials, Operations, Tools & Facilities related to the WO.
Client side Synchronization occurs in the background without a user’s knowledge or input.
The transactions sent from the mobile application use our reliable messaging standards to ensure they reach IFS Applications. A read receipt is sent back to the mobile application for each transaction. The transactions are grouped together for performance to reduce the number of client-to-server calls that are made.
Push Synchronizations are initiated by changes to a database record, which then trigger a Custom Event that is monitoring for changes in that table.
A “Guard condition” is queried which identifies whether the data change is to be sent to a mobile application, or not. If the data is to be sent to mobile a Push Queue record is created. The Push Queue is then processed by the Mobile_Sync_SYS.Process_Push_Data database process, which runs on a high frequency, and runs an Ownership query against all outstanding Push Queue records to establish which mobile users and devices to send the data to.
The results of this query are then updated in the Push Queue. The combination of what data needs to be sent to mobile users, and which users it should be sent to leads to the creation of Synchronization Tasks for each user identified, with the relevant data for that user.
A background job is created to process the "Push" Synchronization Tasks. This background job calls the Middleware Server which will impersonate the Mobile User and fetch the push data. This is done to reference the permission set filters for the user, which ensures only the data the user has access rights to is synchronized as part of the process. The process creates Out Messages and records the data that will be sent to the mobile user, application, device and entity in the Mobile_Sync_Data_Tab table. This optimized delta handling reduces the Synchronization traffic to clients, ensuring the out messages created only contain data that has not yet been synchronized to the client, or that is due to be refreshed on the client.
Out Messages are produced per user, device, application and entity. They will then be processed and only removed when successfully downloaded by the mobile application, as verified by the read receipts received from the mobile client.
Batch Synchronizations differ from push Synchronizations, in that they are processed on a schedule, with a background job checking for a schedule expiry before creating Synchronization Tasks. The Touch App Batch Schedule Database Task, runs on a high frequency (15 minutes by default).
There is a Schedule per entity, which can be set as Daily, Weekly, Monthly, Custom Interval or On Change in Synchronization Rules, with the background job checking the last time the entity was synchronized per user, application, device and entity. If the schedule is changed then it must not be faster than the Touch App Batch Schedule Database Task.
A background job is created to process the "Batch" Synchronization Tasks. This background job calls the Middleware Server which will impersonate the Mobile User and fetch the batch data. This is done to reference the permission set filters for the user, which ensures only the data the user has access rights to is synchronized as part of the process.
The Synchronization Optimizer compares the record the server holds in Mobile_Sync_Data_Tab of the entity data with what is held by the mobile application and creates a “delta”.
Out Messages are produced per user, device, application and entity. They will then be processed and only removed when successfully downloaded by the mobile application, as verified by the read receipts received from the mobile client.
The Grouped Push Synchronization Process is used to synchronize the same set of data to a group of users. This is useful for large data sets where the data is generally always split in specific ways to groups of users, for instance Equipment Objects grouped by Users based on Site access.
A Grouped Push Database Process is created and started when a mobile application with a Synchronization Rule Delivery Method of Grouped Push is deployed. This Grouped Push Database Process then runs continuously waiting for mobile users to connect to IFS Applications. When a mobile users connects the Grouped Push Synchronization Process will fetch a dataset for the Grouped Push Entities based on the mobile users access to the data. The Grouped Push Database Process then manages the maintenance of the fetched datasets per entity. In this regard the Grouped Push Synchronization Process can be split down into two parts, the initial fetching of the dataset by entity followed by the maintenance of the dataset by entity. It is also worth discussin the way the filtering of the datasets is configured & maintained and what happens when a mobile application is re-deployment.
Along with creating the Grouped Push Database Process a special Grouped Push User is also created for each mobile application that is deployed into the environment. For the Grouped Push entities the Grouped Push User must have access to all business roles that are used to filter the data to the mobile users. These business roles could be access to all Companies and/or Sites that will be used by the mobile users running the mobile application.
When a mobile user connects a mobile application for the first time the sync Initialization process is started. This fetches all data from all entities that the mobile user has access to. If one or more of these entities are defined with a Synchronization Rule Delivery Method of Grouped Push then the Grouped Push Synchronization Process is started.
When the Grouped Push Synchronization Process is started for a User/Entity a Grouped Push Filter Map record will be created. The Grouped Push Filter Map for a User is defined by the Grouped Push User Filter which is associated with the entity as shown in Entity Details. If no Grouped Push User Filter is defined for the entity then all records will be returned. In addition to any defined Grouped Push User Filter it is possible to have Permission Set Filter that will further filter the data for the User.
This combination of Grouped Push User Filter and Permission Set Filter defines the Filter that will be used to create a dataset by the Grouped Push process. Once a Filter has been defined for an Entity/User the Grouped Push Synchronization Process will check to see if the same Fitler has been used to previously collect the dataset. If an Initialized Grouped Push dataset exists in the Grouped Push Transaction Queue for the Filter then the Grouped Push Process will return this dataset, plus any subsequent Published changes.
If no dataset exists for the Filter then the Grouped Push Dataset will be Initialized and a record will be created in the Grouped Push Transaction Queue with a Transaction Source of Init. This Grouped Push Transaction Queue will be processed by the Grouped Push Database Process and will result in a dataset being created for the Filter that will be returned.
Note 1: Any Permission Set Filter that is created must not be personally specific for the mobile users as the process that collects the data is executed by the Grouped Push User. The Permission Set Filter also cannot be dynamic across groups of users. If the requirement is to select different data from an entity for different groups of users then multiple Permission Set Filters must be created to select the data required per group of users. The group of users is then defined by granting the Permission Set that the Permission Set Filter is associated with to the required group of users.
To maintain the collected data, database triggers are used based on the Grouped Push Change Detection Tables defined in Entity Details. When the database trigger is executed a Grouped Push Transaction Queue record is created with a Transaction Source of Trigger which is processed via the Grouped Push Database Process.
This process Orders and Groups the data changes with the relevant Grouped Push Datasets. Once the data changes have been associated with the relevant Grouped Push Datasets the changes are Published to the associated mobile users by creating Out Messages.
To ensure the Initialized Grouped Push Dataset remains current the Grouped Push Synchronization Process will re-create it on a daily basis at 1am by default. The default frequency is configured in Application Parameters with the INIT_SCHEDULE parameter. At this time a new record will be created in the Grouped Push Transaction Queue to Initialize the Grouped Push Dataset. This Grouped Push Transaction Queue will be processed by the Grouped Push Database Process and will result in all current Grouped Push Datasets being re-created. The next time a mobile user Initiailzises their app the latest Initialized Grouped Push Dataset will be returned plus any subsequent Published changes.
If the Synchronization Rule schedule for the Grouped Push entity has not been changed, then at the same time the Initialized Grouped Push Dataset is re-created a Grouped Push Transaction Queue record will be created with the Transaction Source of Refresh. If the Synchronization Rule schedule for the Grouped Push entity has been edited then the Refresh process will run on whatever schedule has been set.
The Refresh process compares the data in the Grouped Push Datasets with the current state of data in the entity. If any differences are found then the delta will be Published to the relevant mobile users. The Refresh process is designed to handle changes to the Grouped Push User Filters.
The Grouped Push User Filters are used to determine which Grouped Push Dataset is collected and maintained for groups of users. When the Grouped Push User Filter is changed the data differences are Published to the mobile users via the Grouped Push Refresh process.
The Refresh process compares the data in the Grouped Push Datasets with the current state of data in the entity. If any differences are found then the delta will be Published to the relevant mobile users. The Refresh process is designed to handle changes to the Grouped Push User Filters.
Note 1: After changing the Grouped Push User Filter results, changing permission set security or adding a Permission Set Filter the Mobile Application Cache must be refreshed via the Refresh Server Cache process.
When a Mobile Application is re-deployed the Grouped Push Datasets are re-initialized via the Grouped Push Transaction Queue and the Refresh process is started for all Grouped Push Datasets to ensure all mobile users synchronized data now adheres to the newly deployed mobile applications Grouped Push configuration.
When the Out Message is created via either the Batch Process or the Push (and Batch) Process, a push notification is then sent to the client. This notifies the client that there is new data available to download.
There is error handling capability on the CRM mobile application, but not on mWO or on Time Tracker. This is controlled at design time by the Mobile Error Handling as shown in the Application.
For further information see Troubleshooting Mobile Applications.
On the server side there are three queues that can be checked for transaction errors, the failed transactions queue, the deleted failed transactions queue and the ignored failed transactions queue. If a transaction fails for any reason, transaction grouping is used based on metadata groupings, which means that a failed Work Order for example, will cause all messages related to that particular Work Order to queue as a result of the failed transaction, but will not stop any other messages going through related to other transactions.
For further information see Failed Transactions and Troubleshooting Synchronizations.