In the latest instalment of our Qlik Best Practice Series, we’re looking at creating a reload schedule in Qlik Sense and Chris Lofthouse has provided all the information you’ll need.
Creating a reload schedule in Qlik Sense is important. The scheduling of tasks in Qlik Sense is handled by the Qlik Sense Scheduler Service (QSS). It is therefore imperative that a Qlik administrator understand its limitations before creating a reload schedule. By doing so, it will help devise the optimum reload schedule for your Qlik site, additionally reducing reload queues and potential failures.
Ometis’ Best Practices For Creating A Reload Schedule In Qlik Sense:
- A reload task typically consumes at least one core, therefore it is best practice to create no more than one concurrent task per core. In fact, it is poor planning if this is exceeded. If this number is exceeded then Qlik will most likely queue the additional tasks, and therefore extend the total completion time – taking longer than had you better planned the schedule. This was a particular issue in the Qlik Sense versions predating 3.2, which resulted in tasks failing or not starting when the number of parallel reload tasks exceeded the number of CPU cores on all reload nodes.
Qlik’s official recommendation is n(cores)-2 reloads per reload node. This provides other processes, such as the operating system, enough resources to perform their tasks. The max concurrent reloads is set and configured in the Schedulers section of the QMC. This should be appropriately configured by the Qlik administrator and adhered to by the users with permission to create tasks.
For single node implementations, it is recommended to consider running fewer concurrent tasks to ensure the other services have enough resources to perform their duties also.
Note: Do not forget to consider the monitoring apps and user directory reload tasks when creating the schedule.
- To avoid running parallel tasks and to minimise the delay of reloads starting between tasks, use task chaining. This is more effective than using a random time interval and creates a more optimum schedule.
- Implement an intuitive and scalable naming convention for the reload task schedule. For instance:
- We often prefix a task name with a number block, based on where it lies in the sequence. Those towards the beginning of the schedule may start with ‘10’ or ‘100’. The next block of tasks that may rely on those reloads, or considered less important, may start at ‘20’ , ‘200’, etc. This helps visually group and sort the tasks, it also allows padding for adding new tasks without renaming the entire schedule.
- Avoid using redundant words such as ‘reload’, ‘reload of’, ‘task’. You’re in the tasks section of the QMC, everything here has the same purpose – to reload!
- Instead, give the task the name of the app and optionally include the stream name. For example, ‘100. App name (Stream name)’.
- Set a sensible task session timeout, don’t leave it as the default 1440 minutes (a day).
- Set an appropriate amount of max retries – typically we set this between 1 – 3, depending on the data sources it’s trying to connect to and the importance of the task.
- On time-based triggers remember to set the correct time zone and whether it should observe day-light savings hours.
- For task-event triggers, set an appropriate time constraint and test it! The default is 360 minutes (6 hours).
- Provide an intuitive name for triggers:
- For task-event triggers, we usually specify the task name it’s based on followed by the status it should reach to be triggered. For example, ‘100. App name (Stream name) – SUCCESS’.
- For time-based triggers, specify the time, dates or interval it should trigger. For example, ’15 minute intervals’, ‘Monday @ 02:00’ , ‘Daily @ 06:00’, etc.
Creating A Reload Schedule In Qlik Sense – Our Summary
A Qlik site can operate much more efficiently given an optimum reload schedule, particularly for single node environments. Something less related to the schedule, but still important to manage reloads, is the script logs. In the event of a reload failing, making use of TRACE statements in the Qlik script can help the administrators determine the exact point the reload reached before it failed. If the script relies on variables this can also be beneficial for understanding how they were evaluated at run-time.
Lastly, remember to check the Operations and Reload monitor applications for a better understanding of how your schedule is performing. It can be especially useful for identifying peak times for reloads, failing rates, trends and how to optimise the schedule.
To stay up-to-date with all the latest Qlik tips and tricks please remember to follow us on LinkedIn, Twitter and YouTube.
Topic: Data analytics