|Estimated Configuration Time||30 min|
The Qlik Sense Telemetry Dashboard is a project created and maintained by the Enterprise Architecture team in Qlik’s Americas Presales organization. It leverages additional logging which can be enabled in Qlik Sense Enterprise February 2018 and newer releases of Qlik Sense with some data enrichment with API calls to provide visualization-level performance data for Qlik apps. The goal of the tool is to present this performance data in actionable ways for administrators / developers to have precision when focusing on performance improvements for Qlik apps.
Telemetry in Qlik Sense Enterprise is essentially:
- a threshold based logging. for
- Qlik Engine requests
(1) makes Telemetry different than traditional application logs. ERRORs or WARNs in Telemetry are based on the thresholds the administrator sets.
(2) refers to the fact that the Qlik Engine treats requests differently than what a user might expect. Engine requests can refer to obvious things like opening an app (
OpenApp method) or reloading an app (
DoReloadEx methods). Those are straight-forward. Less intuitive is that when inside of a Qlik app, each visualization on a sheet is a separate request to the Engine. In this example we are high-lighting the 8 different requests (
GetLayout method) which are made when opening our example sheet:
When a request breaches the configured threshold, the individual request will be logged. In the example above, if the filter pane in the upper left hand corner breaches the threshold, the log file will record the request for just that object (e.g. a
GetLayout call for object
The key pieces of information that are logged when a request breaches the threshold that are used in the associated Qlik are include:
| ||Time of the breached threshold|
| ||Level of the breach (ERROR vs. WARN)|
| ||UserId of the user who was making the request|
| ||UserDirectory of the user who was making the request|
| ||Engine API method of the request(e.g. |
| ||AppID where the request occurred|
| ||ObjectId of the object requested|
| ||The type of object requested, if relevant|
| ||How long did this request take to be completed?|
| ||How much RAM did the request take|
This is the extent of the product level changes which were first implemented in Qlik Sense Enterprise February 2018.
In order to increase the consumability and provide context for these logs, the Enterprise Architecture Team has created a project to surface this data in context and in a Qlik app.
The Qlik Sense Telemetry Dashboard project provides an installer which installs a couple of things which will be used in presenting the data in a consumable fashion:
- It installs NodeJS scripts which build out meta-data about apps
- It configures data connections used by the Qlik app
- It imports the Qlik App
- It configures chained tasks which (a) rebuild the meta-data and (b) reload the Qlik app
(1) is needed to build out the hierarchical relationship between visualization objects, the sheets in which the visualizations live, and the apps which hold those sheets.
This hierarchy allows for the consumer of the Telemetry Dashboard Qlik app to place the request in context. Moreover, with this context, specific feedback can be made to the developer(s) of this app to focus their optimization efforts on the offending visualization(s) rather than providing much more generic feedback that ‘the app is slow’.
Since the project needs these NodeJS scripts to build the hierarchy of app <> sheets <> visualizations, the installer configures an External Program Task which runs the NodeJS scripts as a Qlik task. This External Program Task is chained so that its successful execution reloads the Qlik app which depends on output from the External Program Task.
As covered above, this feature in Qlik Sense Enterprise requires the administrator to set the thresholds which register as ERRORs or WARNs. Making the configuration change is covered on the project’s wiki. We will cover what values are appropriate to be set. A common set of configs are:
ErrorPeakMemory=2147483648 WarningPeakMemory=1073741824 ErrorProcessTimeMs=60000 WarningProcessTimeMs=30000
This means that any request which uses over 2GB (
2147483648/1024/1024/1024) will be logged as an ERROR in the log. Requests over 1GB will be logged as a WARN. Likewise, any request which takes over 60s (
60000/1000) will be logged an ERROR and those taking over 30s will be logged as a WARN.
These settings are generally appropriate for most environments.
As a rule, the
Warning values for
ProcessTimeMs should have enough of a gap to cover a steady increase of the underlying values. For example, using
ProcessTimeMs, setting the
Error value to 60s and
Warning value to 59 seconds would not provide enough delta between
Warning to determine whether a particular
Error event was due to a steady increase of the underlying data (as is expected for many Qlik apps) or due to a one-off event. Providing a healthy gap between the
Error values will allow the Qlik administrator and the BI Developers a directional sense of changes over time.
For Production environments, the
Warning thresholds, especially the
ProcessTimeMs value, should be set at a level where the response time is slow enough to at least register. The
Error thresholds should be set at a level where relatively quick attention by the administrator and/or developers is needed.
Once the Telemetry logging is configured, the server has run for a few days to build out a historical record of the requests that breached the configured thresholds, and the Telemetry Dashboard has been installed + reloaded, the administrator and/or developers should review the app for anomalies.
<more stuff to be added later>
- Qlik Sense Telemetry Dashboard Project
- Youtube Video - Initial Configuration
- Youtube Video - Installation
- Youtube Video - App Config
- Youtube Video - Analysis