As well as recent new functionality in both the client-managed (on-premise) and SaaS (cloud) versions of Qlik Sense, Qlik have fairly quietly released a new companion product that could be a game-changer for cloud-first and hybrid Qlik Sense deployments – Qlik DataTransfer (yes, another Qlik Product without a space between the words).
Qlik DataTransfer is described as “a lightweight utility that will securely push your on-premises data and local files into Qlik Sense® SaaS”. I’d debate whether a 450MB installer can truly be termed “lightweight” but it’s definitely a powerful utility and is, as far as I can tell, a custom packaging of Qlik Sense itself – similar to Qlik Sense Desktop – which would explain the install size.
The purpose of Qlik DataTransfer is to address one of the major issues with adopting a cloud-based analytics platform like Qlik Sense SaaS – connectivity to on-premise data sources. Previously, VPNs to private cloud, “holes” in firewalls or FTP/SFTP file uploads needed to be opened to allow access from cloud platforms to on-premise data which was always problematic from a security, functionality and reliability point of view. With Qlik DataTransfer this becomes a seamless experience pushing data from on-premise to the cloud.
When you open Qlik DataTransfer the first thing you need to do is set up a connection to your QS SaaS tenant by providing the tenant URL and an API key – which can be generated by anyone with the Developer role in their Settings as long as API key generation is enabled.
Once you’ve got that set up, you have five areas you can work with:
We’ll look at them in turn.
Here you can connect to a variety of source systems and upload the resulting data as a QVD to your QS SaaS tenant. Here are the options for data sources:
Once you’ve connected to one, you get the familiar data load interface that you’re no doubt used to from Qlik Sense:
You can’t get to the data load editor or manager as far as I can tell so it’s intended for very simple extracts, which means you’ll need to do any data modelling etc once the data has got to your SaaS tenant.
The “Dependent apps” area allows you to automatically trigger a reload of one or more apps when the data connection is reloaded and uploaded.
Once uploaded the data is in a QVD in the specified space – automatically named with the data connection name and table (although if you change the data connection name, the file name updates so watch out for that):
Initially you might think this just uploads a QVF or QVW to the cloud environment, but it’s actually smarter than that. It allows you to schedule reloads of these apps – all within Qlik DataTransfer – and it uploads all the tables from the resulting data model to the SaaS platform as QVDs. You can then trigger reloads in the cloud. This means you can use complicated scripts (maybe created in Qlik Sense or QlikView Desktop) to produce some nice clean data to upload to the cloud for use by your dashboard apps.
Here’s the options you have for the app:
And here’s what happens when you do an upload – which actually reloads and then uploads – in both Qlik Data Transfer and the files as they appear in Qlik Sense SaaS. In this case, the reload failed but it still uploaded the tables that were in the app:
This area allows you to set up folders that should be monitored for file updates – handy if you have automated reports dropping CSV or Excel or other extracts into a folder. Again you have the option to trigger reloads of apps once the data has been uploaded.
This just lists the data sources you’ve set up in data connections – the connections are the actual tables etc you’ve selected to extract in a batch – which is somewhat confusing. I think what lives in “Connection details” maps more closely to “Data Connections” in Qlik Sense Enterprise and “Data connections” is more like “Data extracts” or “Data batches”. However, it’s useful to be able to see and manage the underlying connections separately from the extracts.
For Data folders, files are automatically uploaded when they are added or updated to the folder. However, for Qlik apps and Data connections, schedules have to be set. These have all the options you’d expect.
It’s a game-changer! Obviously it’s not as advanced as something like Qlik Replicate, but for most organisations it would have sufficient functionality to allow them to move cloud adoption forwards.
Along with recent changes in Qlik Sense SaaS – the removal of app reload and data storage limits as well as the increase of per-app in-memory allowance from 2.5GB to 5GB – Qlik DataTransfer successfully and easily bridges the gap from on-premise to cloud. New and existing deployments of Qlik Sense should consider whether a cloud deployment could be more reliable, performant and cost effective for them. If you’d like us to help you work through your options for this then please drop us a line!
If you’d like to find out more about getting the best out of Qlik, including a masterclass in accelerating insights through augmented analytics, there’s less than two weeks remaining to register for Qlik Analytics Tour 2020 hosted by Ometis. You can register for FREE here.