Back at the start of August 2019, Qlik made available the Qlik Data Catalyst Closed Access Product Accreditation, something I jumped at the chance to take and I’m glad I did! For those who may have never heard of Qlik Data Catalyst, it is Qlik’s enterprise data management solution, and a previously-missing piece for enterprise data governance in Qlik’s data platform.
In the modern age of analytics, data sets are becoming larger and broader. Data is being captured both from internal and external sources and demand for access to that data is being driven from business users from the top to the lowest levels of an organisation and even external individuals. Guaranteeing data integrity and security, while still maintaining availability and usability, has never been more business critical. For organisations to manage and govern data effectively, a data management solution, such as Qlik Data Catalyst, is essential.
Qlik Data Catalyst (QDC) offers the ability to:
- Onboard data: Data ingest, validation, quality and profiling, from all sources and formats.
- Catalog / Discover: Search, preview and enrich data, metadata and data scores (quality, operational and popularity).
- Prepare: Combine and transform data, from raw to ready.
- Publish: Consuming data in the visualisation tool of choice.
For me, the real power of QDC is the Catalog, as that enables true self-service from the data warehouse.
The catalog, with its Amazon-like shopping experience, provides a graphical interface for business users to explore data sets, and the metadata associated to them, in a tile or list view. If the user finds an item they are interested in, they can choose to view its details, quickly accessing the lineage, field statistics, sample data and description. Similar to Amazon’s recommendation engine, it will also provide related data sets:
In addition to this, business users can enrich the catalog further by adding tags and comments (much like professional shopper reviews) – denoting things like quality, completeness, target audience, constraints and usability of the data. Once the user has found a desired data set they can simply add them to their Cart, and publish the data to their target of choice – mine being Qlik Sense, of course.
Publishing to Qlik Sense is seamless, in fact the entire process for a business user is so intuitive I’m certain my five-year-old nephew could figure it out.
So, what powers the Qlik Data Catalyst Smart Data Catalog?
In summary, Qlik Data Cataylst sits on top of data storage and compute platforms such as Hadoop, Linux FS, Amazon Web Services (AWS), Google Cloud Platform and Microsoft Azure – leveraging their enterprise-grade scalability, reliability and performance.
Data can be onboarded in one of three ways:
- Managed: Data and its metadata is fully stored and managed by QDC.
- Registered: Only the metadata and sample data is stored and managed by QDC, the actual data remains in the source system.
- Addressed: Only metadata is integrated into Qlik Data Catalyst.
If necessary, users can perform powerful transformations to turn raw data sources into business-ready data, without writing a single line of code, complimentary of a simple graphical interface.
The heart of its power is in the metadata, however. Metadata is harvested from the source system to begin with. Additional statistics are captured throughout the dataflow (validation, profiling, lineage, usage, Personally Identifiable Information detection etc.) and importantly complimented by the business using tagging and comments.
All this combines to provide users with enterprise data on-demand.
Qlik Data Catalyst for QVDs
What excites me more was the announcement of Qlik Data Catalyst for QVDs, at Qonnections earlier this year. QDC for QVDs, is a lite-version of Qlik Data Catalyst targeted for release in the second half of 2019. For those unfamiliar with QVDs, a QVD (QlikView Document) is a highly compact data file that is optimised for reading into Qlik Sense and QlikView, see our post on their benefits here. Qlik Data Catalyst for QVDs will hopefully provide the smart data catalog capabilities of QDC over a QVD layer. In short, for companies already invested in Qlik Sense, this will enhance their data platform by providing enterprise data governance and improved self-service capabilities.
By the way, if you were wondering if we passed the accreditation, we did. This means we at Ometis can officially talk to you about Qlik Data Catalyst and how it can benefit your company. If you want more information, please get in touch today.
By Chris Lofthouse
Follow @clofthouse89
Comments