Schedule Distribution Service¶

This service is optional and should only be used when required, based on the datasets to be scheduled. The purpose of the Schedule Distribution Service is to allow for the efficient scheduling of very large scheduling problems.

Modes of Use¶

The Schedule Distribution Service (DST) has two distinct modes of use: Segmentation mode and Feeder mode. These can be used independently or in combination.

In either mode, the Schedule Distribution Service will create internal 'SEGMENT' or 'ATOMIC' datasets which are then processed by one or more Dynamic Scheduling Engines (DSEs). The management of these internal datasets is handled internally by the DST and DSE(s). No changes are required to the external application, or to any other scheduling components.

Feeder Mode¶

Feeder mode is recommended when there are a large number of activities that can potentially be scheduled, but only a small percentage of the available activities are expected to be allocated in the plans produced. This is typically used where customers have a high volume of planned maintenance work with long time windows of availability, alongside some short term reactive work.

When using this mode, the DST will create a single internal dataset to be processed by the DSE. It will then intelligently select which activities to feed into this dataset to be processing by the DSE. It will then exclude from this dataset any activities which it considers are highly unlikely to be scheduled, so that the DSE views a much smaller subset of the total data.

The DSE will schedule the segment dataset as normal, and produce plans. The DST will then read the plan produced by the DSE, and review its decision about which activities should be included in the DSE dataset. Where appropriate activities are then be added and/or removed from this dataset.

When input changes are detected the DST will ensure the DSE dataset is also updated as appropriate.

When determining which activities to send to the DSE, the DST will assign each activity to one of three categories:

1. Those which must be sent to the DSE.
2. Those which will never be sent to the DSE.
3. All other activities, which are sent depending on their level of 'importance'.

Must Send Activities¶

Activities will always be sent to the DSE if they have any of the following characteristics:

• Activities that must be scheduled, either because they are at status committed or higher, or because they are fixed to a resource and time.
• Activities which have the attribute 'reactive' set to true in the input data. This is an attribute on the Activity and Activity_Type entities - see the Scheduling Schema guide for details.
• Activities with a jeopardy deadline within a define period from the current time. This uses the parameter 'FeedActivityInJeopardyThreshold' which defaults to 4 hours.
• Activities of class DEPOT.
• Activities with no location.
• Activities with invalid data (to allow the DSE to raise the appropriate exceptions).

In addition, activities will always be sent to the DSE if they are linked (e.g via a co-requisite or pre-requisite) to another activity which is being sent to the DSE. The same applies to activities at the same location or locality.

Finally, any activities that are allocated in the current DSE plan will not be removed from the feeder dataset.

Must Not Send Activities¶

Activities will never be sent to the DSE if they are at status do no schedule, or if they have no availability (and if they do not fall into the previous category).

Importance¶

For all other activities the decision is based on a measure of the importance of each activity. The importance is based on two primary factors:

• Value. This is a measure of the maximum value of scheduling the activity within the current scheduling window - allowing for variables such as ageing factor.
• Location. The importance of an activity is increased if it is near to other allocated activities.

The DST will then send the most important activities to the DSE. The number of activities sent depends on the number of activities that the DSE is able to schedule, and the value of the parameter 'FeedActivitiesMultiplier', which defaults to 0.5. This number represents the approximate target number of unallocated activities within the DSE dataset, relative to the number of allocated activities.

As an example, if the DSE has allocated 8,000 activities then using the default setting of 0.5 the DST would aim to have 4,000 unallocated activities in the DSE dataset, so 12,000 activities in total. If the setting is changed to 0.25 then the DST would send 10,000 activities to the DSE, with 2,000 expected to be unallocated.

Feeder Aggregation Mode¶

The feeder aggregation mechanism uses a similar approach to the exclusion mechanism, except that:

1. Activities are only considered to be important if scheduled in the dynamic scheduling window.
2. Unimportant activities are aggregated instead of being excluded.

Activities are only aggregated is certain conditions are satisfied:

• The dataset is large enough to require it, based on the parameter 'AggregationTargetActivityCount' (default 10,000).
• The activities to be aggregated share the same basic characteristics (skills and regions required and activity type).
• The activities to be aggregated have similar values, availabilities and SLA times.
• The activities to be aggregated are not too far apart, based on the parameter 'AggregationMaximumMergeDistanceMetres' (default 2,000).
• The aggregated activity duration would not be too long, based on the parameter 'AggregationMaximumActivityDuration' (default 2 hours).

The allocation of aggregated activities is returned in the plan output. In the Allocation rows the activity id will be the id of the aggregated activity, and the details of the aggregated activity as provided as Aggregated_Activity rows and Aggregate_Member rows. See the Scheduling Schema guide for details.

Segmentation Mode¶

The DST segmentation mode is designed for use with very large scheduling problems. This is for problems which would be considered too large for a single DSE to schedule, and which are not appropriate for using other approaches such as the DST feeder mechanism.

In this case the DST will create multiple internal datasets to be scheduled by multiple DSEs. The number of DSEs required will depend on the size of the problem. The internal datasets will be distributed amongst the available DSEs in the same way as standard datasets, allowing for both the size of the dataset and the capability of the hardware each DSE is running on.

The DST will decide on an initial segmentation of the data, by evaluating the data and considering factors such as regions, availabilities, and the geographic distribution of the data. As input updates are sent to the system the DST will ensure the internal DSE datasets are also kept updated, and review the current segmentation. If substantial changes have accumulated which significantly change the distribution of data then the DST will change the segmentation accordingly.

The choice of segmentation method used is determined by the parameter 'PermittedSegmentation'. There are three methods of segmentation currently considered:

The DST can also consider multiple segmentations methods, and use the methods in combination. Simply select the options you wish it to consider, and it will use whichever combination is the most appropriate.

DSEs can be added or taken off line in a seamless manner to increase the computing power or reduce it for maintenance upgrade purposes. The computing load is re-distributed across the servers automatically, whilst maintaining the defined segments.

For 'STATIC' scheduling problems, the DST can also queue up the segment datasets to be processed. This means that a single DSE in combination with a DST can schedule a very large static problem, without the system being overloaded.

Update Handling and Re-segmentation¶

When the DST is running in segmentation mode and input updates are received, the DST will attempt to update the existing internal datasets. Occasionally it may be necessary to re-segment the data. This will be due to one of the following reasons:

• The changes to the data mean that the current segmentation is no longer valid. For example, if constraint segmentation is used and a change means that two or more segments need to be combined.
• The volume of schedulable activities in the input data has changed by more that the 'ResegmentationFactor' parameter (default 50%).

If re-segmentation is required, then once new segments have been determined, the DST will update the existing DSE segment datasets and/or send new load inputs for the datasets as appropriate.

Multiple modes¶

The DST can be set up to use both Feeder and Segmentation modes at the same time (though this option is not available with the Feeder aggregation mechanism). In this case the DST will first apply the feeder mode to exclude a number of activities. If the resulting dataset was still large, it would then consider the permitted segmentation options to try to split the data into multiple internal datasets.

Atomic Datasets¶

Sometimes for a given dataset the DST may determine that no feed or segmentation needs to be applied to the dataset, and the DSE can simply process the data directly. In this case the DST will create a single 'ATOMIC' dataset for the DSE to run against, which simply redirects the DSE to processing the parent dataset. The advantage of this is that the DSE can then process the data with virtually no overhead compared to running without the use of the Schedule Distribution Service.

Deployment¶

The following guidelines can be used to determine if the DST should be deployed on a system, and if so which mechanism should be used.

Performance Metrics - Feeder Mode¶

The graph below uses a dataset with 50,000 activities, of which around 6,000 will be scheduled, and compares the output from using DST feeder mode with that of using a DSE only.

The points on the graph represent where plans were returned. The horizontal axis shows the number of minutes since the problem was first submitted and the vertical axis the number of activities that were allocated in each plan.

It is clear that the use of the DST feeder allows plans to be generated sooner and of a higher quality in this case.

Performance Metrics - Segmentation Mode¶

The graph below uses a number of datasets of varying sizes, and measures the time taken for a first plan to be produced. The graph compares performance with a standard DSE setup (the blue line) to that with a DST and a given number of DSEs in segmentation mode.

The points on the graph represent when the first plan was returned in each case. The horizontal axis shows the number of activities in the dataset used. The vertical line shows the time taken for the plan to be produced.

The graph shows that for datasets with more than 15,000 activities it will often be beneficial to use the DST in segmentation mode.

It should be noted that the relationship between problem size and number of DSEs becomes linear when using the DST segmentation mode, thus enabling the system to scale to very large scheduling problems.

Handling Large Datasets¶

15,000 activities is the recommended maximum size for a DYNAMIC dataset running on a high-spec server. This may be treated as an aggregate limit, so a Dynamic Scheduling Engine running on a high-spec server may process one 15,000-activity DYNAMIC dataset, or three 5,000-activity datasets, or six 2,500-activity datasets.

The Schedule Distribution Service may be used in various configurations to handle datasets larger than 15,000 activities in certain circumstances, as follows:

Deployment Decision Tree¶

The diagram below shows the general guidelines that should be used when deciding is the DST is required. The assumption is that all activities are included in a single scheduling dataset.

If the DST may be required, see the sections above to determine which modes of operation should be used.