There are several actions on both the server side, and the client side, that can be used to effectively troubleshoot Synchronization issues. These issues may vary, but typically involve the following:
- Data not being received by the client app - a download issue
- Data returned from the client app not appearing in IFS Cloud - an upload issue
- Slow Synchronization times
The troubleshooting may need be done on both client side as on the server side. The main focus in this guide will be on the server side as there are not many investigation that can be done on the client side.
To troubleshoot synchronization issues you you need to determine if it is
- A download issue (from backend to device)
- An upload issue (from device to backend)
- Slow performance
Furthermore you need to determine where the problem lies, if it is on the server or in the client.
For download issues you need to check the following
- Out Message Queue
- Synchronization Tasks
- Background Jobs
- Push Queue
- Database Processes
- Synchronization Traces
For upload issues you need to check the following
For Slow Performance investigation you need to check the database with the included sql statement.
The upload issue, Failed transactions is covered in the troubleshooting guide Failed Transactions. Group Push is covered in the troubleshooting guide for Troubleshooting Group Push. The rest will be covered in this troubleshooting guide.
If it is a download issue it is best to start with the Server Investigations before continuing with Client Investigations.
If it is an upload issue it is best to start with Troubleshooting Failed Transactions before continuing with Client Investigations. Is is more likely that you need to perform Client Investigations for an upoad issue then for a download issue.
Server side investigation is mainly to check and track where it goes wrong in the backend system.
Out Message Queue¶
The Logs will confirm the production of an Out Message, and it is also possible to look in the Out Messages queue and view the message that has been created . You can also select a specific Out Message and view the data that is being sent to the Aurena Native application command Message Body Details. Out Messages are created as either insert, update or delete messages, and once processed you will see the results in the Aurena Native application data.
Do you have an out message or not?
- If it turns out that everything is OK on the server then you need to continue with Client Investigations.
- If you do not have any out messages than it means that the out message has not been created at all or it has been sent succesfully to the client. Best way to determine what of the two option it is, is to continue with Server Investigations. Next step to check is the synchronization tasks.
In terms of troubleshooting Synchronization issues, it is also possible to check the Synchronization Tasks list in IFS Cloud, which will provide information on how long it is taking to run Synchronization Tasks and produce the data by execute the task. Each task has linked Synchronization Service Queue rows which will be process by Synchronization Server. This process will create the actual out message that will then be sent to the device. In the Sync Tasks list you can see if a task has completed, and get information on how long that task took to run for all entities listed.
Has the synchronization task been completed?
- If yes, then an out message should have been created and you should continue with Client Investigations.
- If no, then you need to check that the Synchronization Service Queue status.
The Background job that has been created to produce an Out Message with Batch Data for an Aurena Native. This is done by the scheduled task in each Aurena Native App. The progress column in background job screen shows the number of sync tasks created after executing this Background job.
Has the background job been executed?
- If yes, but you have an error message. Try to resolve the error message and re-run the job.
- If no, you need to check why the background job has not been executed. It could be stuck due to other long running jobs and are waiting for an idle session in order to proceed.
- If yes, and no error message, then you need to check the push queue.
In the Push Queue, it is possible to see the creation of push queue records for every IFS Cloud database entity that is changed, and is to be synchronized to device.
What does the status message say? Has the push queue been executed and are completed?
- If the push queue is in Completed state then everything is ok.
- If the push queue is not in Completed state, then you need to check the Database Processes.
It is possible to examine the Background Process that produces Push Queues and Synchronization Tasks to ensure it is working correctly in IFS Cloud (Mobile_Sync_SYS.Process_Push_Data). The process will indicate whether there have been any failures to process data, Time last run, and time taken to run the process. This would also help in identifying any slowdown in database processing as an impact on Synchronization delivery.
Has the background process been executed?
- If yes and there is no error messages then everything is fine,
- If yes and there is error messages, check what the error message indicate and try to solve the issue.
- If no, check why it has not been run. Is it scheduled to be run at certain times.
Next step, if it is no error found so far, is to check a couple of Logs as well as starting Synchronization Tracing.
Check the following logs when a synchronization issue occurs.
It is possible to view the Logs related to a sync task, which provides a further level of information in relation to the sync task. The user/device logs will tell you when processing started for a specific entity in the sync task, and when processing finished. This means that it is possible to troubleshoot whether custom changes to a database entity are affecting the Synchronization process, which might be visible if for example the changes are increasing the processing time for a specific entity, compared with other elements of the sync task.
We also write to the logs if a single entity sync takes more than “x” seconds to complete. This is configurable in System Parameters and provides additional visibility of synchronization issues that are connected to issues with an entity and related customizations.
It is possible to view Synchronized Entities, which will provide information on when an entity was last synced, and the time taken for the last sync.
Permissions to run Synchronization Traces are enabled when the user is granted the FND_AURENA_NATIVE_SYNC_TRACE permission set, which grants the user the system privilege of “Debugger”. With this in place it is possible to setup tracing for a single device in IFS Cloud. This can be found in either Aurena Native User Details or in Installed Apps/Devices which will generate a trace including all the information about a client connecting, and all the database processes running in relation to that client, including the PL/SQL Statements that are being run. This can provide a large amount of information in relation to specific client issues. Given the amount of data this generates, it is important to run the Schedule Task to clean-up Synchronization Traces if this is left running for a long period. Also, due to the volumes of data generated by a trace, enabling tracing for all devices in a large device community is NOT recommended. It is recommended that tracing is set for devices with issues only.
Device tracing can be found in Synchronization Traces in IFS Cloud:
When examining the trace logs using a key word search is the best way to find issues that have been recorded. This could involve searching for a key word of “error” for example, or searching on an entity name that you know is causing issues.
If no errors is found here the issue is to be found on the client side and Client Investigations needs to be performed, but before you do that please check what you have as System Parameters.
There are a couple of System Parameters that can be set in relation to synchronization processes.
- Aurena Native Queue: This sets the Batch Queue ID for the Aurena Native Batch Synchronization.
- Aurena Native Sync Warning Threshold (seconds): This parameter allows you to define a threshold time for a single entity to be synchronized before a warning is written to the Logs.
There are some limited troubleshooting steps that can be taken on the device to look at Synchronization problems:
- It is possible to examine the local database on the client. This may help to identify any data format related issues that could be having an impact on client side Synchronizations.
- It is also possible to look at the client app logs, to see if any errors are being recorded on the client side, that may be affecting Synchronization processes.
- Examining the status of the Sync Monitor - is it possible to connect to the server, or not? Are there pending transactions showing in the Sync Monitor?
Slow Performance Investigation¶
If you feel that you have slow performance, that it takes long time to do the synchronization or initialization, then you need to check if you have large data volumes and also pertorm test initialization to a "dummy device".
It is possible to perform the option Test Initialization from Installed Apps/Devices that will create a new "dummy" device for the select User, App and App Version and perform an initialization. On selecting the Test Initialization option a prompt will be shown to navigate to the Background Job that will process the initialization request.
Note 1: The "dummy" device will not be included in the batch sync process. This is for testing the performance of the initialization request for a single user.
Note 2: It is best practice to remove "dummy" devices using the Delete App/Device option when the test results have been analyzed.
Checking Synchronization Data Volumes¶
The check of the data volumes for an entity or a user is done in the screen Synchronized Volume or in the lobby page for Aurena Native App Status. You can check for push/batch entities as well as for grouped push entities.