Data Sizing¶
This section discusses the options available when using the scheduling functionality within the scheduling system, depending on the size of the data to be scheduled.
Each independent problem to be scheduled is sent to the system as a separate 'dataset', and various options may need to be considered depending on the size of each dataset, and the type of scheduling required.
Note
The system is designed to handle multiple datasets being processed at the same time. Every component can be installed on multiple instances running against the same system, and each instance is itself capable of processing multiple datasets. The instances will automatically load balance based on the size of the datasets to be scheduled and the hardware capabilities of each running instance.
Note
In addition, it is possible to install multiple instances of the Schedule Query Manager and have each process the same dataset. This feature is enabled by default, and can be controlled by the parameter "AllowMultipleInstancesPerDataset". This is especially useful is there is a single very large dataset which many workbench users will be accessing simultaneously.
Dataset Size¶
The main factor in determining the size of a dataset is the number of activities that require scheduling. This figure should include all 'schedulable' activities, i.e. any activity which the DSE is expected to choose the optimal position for in the schedule.
- It does not include activities at status committed or greater, or activities marked as 'do not schedule'.
- It does not include private activities, or any activities fixed to a resource and time.
- It does include breaks, provided they are not committed or fixed to a certain time.
In addition, the size is affected by the complexity of the schedule. This includes the following considerations, some of which will make the scheduling problem harder, and some easier:
- The number of activities that are linked to one another via co-requisites, pre-requisites or other means.
- Restrictions on which resources are able to carry out activities due to skills, regions and other constraints.
- Restrictions on the scheduled time of activities due to availabilities, SLAs etc.
- Use of parts and depot functionality.
- The geographic spread of the data.
- The number of activities that a resource will typically carry out in a single period of work.
In the following sections we consider what size of data can be handled using a standard system setup, depending on the type of scheduling required. Then in later sections we describe the other options available for very large scheduling problems.
Note
Please note that the use of the Schedule Distribution Service feeder mode and the use of an appointment window may be appropriate even for standard sized data. See the relevant sections of this guide for details of when these should be used.
Dynamic Scheduling¶
Real time dynamic scheduling involves running a dataset continuously, with regular updates to reflect changes made in the real world. The DSE is expected to react quickly to changes and adjust the schedule as appropriate, while constantly searching for improvements.
A dataset which includes up to 5,000 schedulable activities can normally be handled by a single DSE using the standard system architecture. This is regardless of the complexity of the data.
For datasets of between 5,000 and 15,000 activities a standard system will still usually be sufficient, but this may depend on the data complexity. Highly complex problems may require an alternative mechanism. See the section on data complexity to establish this, and if necessary speak to an IFS representative.
For datasets of between 15,000 and 25,000 schedulable activities a standard system may be sufficient, provided the data is not complex, and updates are not required too frequently. It is usually advisable to speak to an IFS representative in this circumstance.
For datasets of more than 25,000 schedulable activities an alternative approach will nearly always be required. Further to this, it is not recommended to use the PSO Workbench to view datasets with more than 25,000 activities as the performance and responsiveness of the interface would be reduced.
The above assumes that the DSE is running on a dedicated machine (or virtual machine with dedicated processors) with a high specification. See the hardware specification section for further details.
Warning
When processing large data volumes it is recommended that command time-outs are added to the connection strings of each component. This is to avoid database time-outs occurring when reading data from or adding data to the database. To set the command timeout to 5 minutes, simply add the text 'Command Timeout=300;' to all connection strings. This applies to both Oracle and SQL server databases.
Warning
For large datasets, object groups should always be used to limit the amount of data displayed to any one user.
Groups can be created within the administration workspace on the Scheduling Workbench, and placed into a hierarchical structure. Each user can also be added to one or more groups via the administration workspace.
Groups can then be linked to resources and/or activities within the input data, and users will only be able to see data for resource or activities matching the groups they belong to.
Using region based filters will also limit the amount of data sent to each client.
Static Scheduling¶
Static scheduling involves running a single one-off schedule on an occasional basis (e.g. once per day or once every few hours). Since the acceptable response time is usually longer than with dynamic scheduling, it is possible to run larger sizes of data for this type of scheduling.
As such, the sizes described in the previous section for dynamic scheduling can generally be increased by around 2-3 times when only static scheduling is required. A maximum of around 50,000 activities can be processed by a single DSE instance, depending on the complexity of the problem.
Note
It is not possible to use static scheduling for datasets where appointment booking is required.
Note
Using the PSO Workbench to view schedules larger than 20,000 activities is not recommended as performance and responsiveness would be reduced.
Reactive Scheduling¶
In some cases a scheduling problem may fall between the 'dynamic' and 'static' categories described above. That is to say that updates to the problem will be infrequent, but there may be small clusters of updates within a short space of time, before a much longer gap until the next schedule is required.
In this situation reactive scheduling can be used. This means that the scheduling engine will respond to updates when they occur, but pause and even unload the dataset after a period without any updates.
This method has the advantage that a single server can handle similar volumes to when running static schedules, but will still be able to process changes as they occur.
Note
It is possible to use an appointment window when using reactive scheduling. If appointment booking is required, it is advisable to stop reactive datasets from being unloaded by changing the parameter 'UnloadReactiveDatasetsAfter' to a high value. This is to ensure that the appointment booking engine can respond promptly to appointment requests.
Note
The DSE will continue to process REACTIVE tasks as a background activity when there is nothing more urgent to be done. This means that if all datasets are REACTIVE, then the DSE will prioritise those with recent changes where the DSE still needs to respond with a new plan. However once all datasets are up to date it will cycle around the datasets and continue to optimise them.
If this behaviour is unwanted it can be turned off via the parameter 'BackgroundProcessReactiveTasks'.
WISE Scheduling¶
The WISE is a tool for long term strategic planning. When using the WISE it is possible to request schedules in two different ways.
The first is simply to produce a one off schedule using a selected demand and resourcing. In this case the DSE will produce a single static schedule, and the guidelines in the previous section for static scheduling will apply.
The second option is to ask the WISE to consider adding or removing resources. In this case it is usual to expect the schedule to take around 20-30 times longer to be produced than with the first option. Thus if a schedule takes 30 minutes to be produced without allowing the WISE to make resourcing changes, then running the same schedule with changes allowed would become an overnight process.
Note
Since the WISE is intended for long term planning it may be necessary to use large data volumes. Where this is required (e.g. > 10,000 activities) the data should be presented in .csv format, and the WISE will then load the data sequentially, and automatically aggregate the data while loading. In this way the problem size is dramatically reduced, and the WISE will be able to provide responses in reasonable time.
Data Complexity¶
The size of data that can be scheduled and the time required to produce schedules is heavily influenced by the complexity of the data. The scheduling system supports a wide range of functionality. Heavy use of some of these features will make the system less responsive compared to processing standard data of a similar size.
The requirements that are most likely to affect performance are outlined below:
- Activity Links: Links between activities (such as pre-requisites and co-requisites) will usually have a negative impact on performance. The impact will depend on the number of activities with links and the size of the chains created by the linked activities.
- Parts: Heavy use of parts can increase the time required to validate the schedule and hence increase the time for schedules to be produced, especially if depots are also used.
- Split Activities: Use of split activities can have an overhead on the time taken to produce schedules.
- Multi-Day Scheduling: If the scheduling window is quite long and each resource has a fairly large number of shifts it is likely that schedules will take longer to produce.
- Under-Capacity: If there are many more activities in the data than are able to be scheduled to the available resources this can cause schedules to be produced more slowly using the standard set-up. The Schedule Distribution Service Feeder mode may well be appropriate here.
- High Activity to Resource Ratio: A typical scheduling problem may have up to around 20 activities per resource. Schedules where the ratio of activities to resources is significantly higher than this may take longer to process.
The Appointment Window¶
If the Appointment Booking Engine (ABE) is to be used to book appointments, it is often necessary to have a longer scheduling window or several weeks or months. This is to allow appointments to be booked as far into the future as required.
This can impact the DSE performance when running in standard DYNAMIC mode, and so it is recommended that a split scheduling window is used.
The first part of this window can be relatively short, perhaps a few days or a week. In this part of the window the DSE will operate as normal, using the standard algorithms to optimise the schedule.
The remaining time forms the appointment window, and this can cover a much longer period of time. In this period the DSE will initially use a simpler algorithm to produce an acceptable schedule, without using up too much time. This allows the DSE to spend the majority of time optimising the first part of the scheduling window, where a fully optimised schedule is required.
Activities which cannot be scheduled within this initial part of the window will not be considered, and as such this allows the DSE to handle a larger volume of activities than would normally be possible. The standard recommended limits should only be applied to those activities which are able to be scheduled in the first part of the scheduling window.
Note
Although the DSE will initially only look to optimise the dynamic part of the window, it will also optimise the appointment part of the window as a background task. It will only carry this out when all broadcast targets have been hit and there are not input changes still to be processed. This behaviour can also be switched off via the parameter 'BackgroundOptimiseAppointmentWindow'.