Traditional ETL platforms have been handling data following this ‘extract, transform and load’ model. Cloud adoption in banking and financial services, has introduced newer cloud ETL tools that work in the same way, only use an ELT model instead and load in a cloud environment.

The growth of data has brought a number of new roles within an organization such as data engineers, data analysts, etc. All these data consumers require a data platform that supports their needs and unique requirements while promoting collaboration, reusability and extensibility of data pipelines, knowledge sharing on data and data preparation.

Flexibility, Scalability and Reuse

Validata ConnectIQ has taken classic ETL to a new level, through a data-centric architecture that adds enterprise scalability, reusability, flexibility, high performance, and data governance, to the classic capabilities of ETL.

The added object-oriented abstraction layer enables reusability of components, data pipelines and logic, along with resulting data sets. Complex data can be transformed into a flexible and reusable form that data analysts and scientists can use. Through self-service secure data sharing, data consumers can get involved and perform their own incremental transformation on the data as and when required.

Simple, Easy to use Interface

Unlike traditional tools that use coding or have an old data flow interface with more complex interfaces for each component, ConnectIQ has a no-code, simple, drag-and-drop graphical interface for creating and managing your data pipelines. This easy-to-use UI is like excel worksheets, allowing users to see and profile the data at every step. These worksheets can be shared with the team for customization of the pipeline or extension of the pipeline with other pipelines depending on their access rights. The extension pipelines can then be attached to the components within the originating pipelines.

Analysts can combine multiple data sets, enrich and slice them in different ways to shape the data to the specific needs. For example, we can blend together data from a CRM (Salesforce, SAP etc), website, sales system and Temenos core banking, cleanse it and enrich it with calculated columns to produce a customer activity data set. Then we give access to the output data set and viewing access to the pipeline which can then be re used and extended downstream by the rest of the community; data stewards, data engineers, analysts and data scientists.

Analyst and Data Scientist Functions

ConnectIQ also includes a rich library of functions, so that an analyst can extend a data pipeline, such as:

• Join and combine data in different ways
• Sorting and filtering of data
• Group and aggregate data to slice it in advanced ways
• Extract meaning from text fields or add context etc

The same also apply for data scientists in order to shape and organise data with extended data pipelines according to their needs, but have additional specialized functions such as:

• Encode data to feed AI and ML models
• apply advance algorithms to enrich and shape data, etc

Data-centric Architecture

Validata ConnectIQ is a cloud-native dynamic data platform for banks and financial services organisations, built on a data-centric architecture that embeds security and trust directly into data, accelerates data delivery, facilitates secure data sharing and enables rich data insights with data integrity at the core.

It adds an integrated, data-centric layer of security controls for all sensitive data sources, keeping data protected as it moves in and out of storage systems or applications as well as changing business contexts, regardless of the network or application security.

Through automation, it simplifies and enables data governance, and keeps track of traceability of changes in your data with version control. “Time Travel” allows you to issue queries against any moment of time and instantly retrieve the data as of that instance.

In addition, the platform is built with normalized and dimensional modeling techniques, it extracts and self-updates metadata and production data from your applications, and creates a normalized database for faster search and quering.