Authored by: Ravi Shankar, senior vice president and chief marketing officer at Denodo
Cloud technology is making it possible to maintain business continuity and provide continuous learning during the current COVID-19 health crisis. It enables people to remain connected despite city lockdowns and stay-at-home efforts.
Today, many organisations use more than one cloud and storage service. In a single network architecture, they may tap into two or more cloud systems from among the four primary platforms Amazon Web Services (AWS), Google Cloud Platform, IBM Cloud, and Microsoft Azure. These platforms all have different architecture and provide different applications.
Most organisations began their experimentations with cloud computing by first running part of their applications on cloud and the remaining on-premises. In other words, they took a hybrid cloud approach, operating cloud computing environments that combine on-premises, private cloud, and third-party public cloud. Multi-cloud is simply an extension of that approach, whereby organisations run some of their applications on-premises and others on different cloud environments, tapping on services from different public cloud service providers.
Naturally, organisations select cloud providers based on the applications they already have in place. Very likely, organisations chose Microsoft Azure because they were using Microsoft Dynamics, Office 360, or SQL Server. Others might opt for Google Cloud Platform as they run data science applications, or they might go for AWS as they are using S3 storage.
Multi-cloud strategy enables organisations to customise their technology infrastructures in a way that suits their own unique business processes. It enables organisations to benefit from the advantages that each platform brings and minimise the impact of their shortcomings.
However, multi-cloud architecture is not without its challenges.
The most critical challenge in a multi-cloud environment is that data stored with one cloud service provider will be siloed from the data stored in another. This means that when organisations initially set up a multi-cloud environment, they will not be able to gain a holistic view of their data. The traditional data-integration strategy, in which data from the multiple cloud systems is transferred to a single repository, such as a data warehouse, using extract, transform, and load (ETL) processes, is complex and time-consuming, and because they are batch-oriented processes, they cannot integrate the data in real time.
Additionally, though cloud service providers provide very good security within their own environment, security can be an issue when organisations have to access disparate data across the different clouds. There is always the risk of exposing the data to unintended users.
Embracing Heterogeneity in a Multi-Cloud Environment
To overcome the first challenge, organisations will need to deploy a robust cloud data integration solution. Data integration tools have reliably delivered integrated data from systems like Accounting and Payroll, CRM, Enterprise Resource Planning (ERP), etc., to specific destinations such as a data warehouse, for years.
Currently, with the growth of cloud, data integration has also evolved to integrate data from on-premises and cloud systems. Modern data integration and data management solutions, such as data virtualisation, can efficiently integrate data spread across public and private clouds, and deliver the integrated data to systems that reside either on-premises or within the cloud environment itself.
Unlike traditional data integration solutions, which rely on data replication to move data from disparate systems into a consolidated repository, data virtualisation provides real-time views of the data in its existing locations.
Data virtualisation also overcomes that second challenge, regarding security. Because data virtualisation is implemented as an enterprise data-access layer, it provides a natural gateway that enables organisations to manage security protocols across multiple cloud systems from a single point of control. With data virtualisation, access to data is limited to the intended end users only in line with their authentication levels.
Modern organisations should be able to freely choose where they store their data and run their applications, be it on-premises, in the cloud, in a hybrid mode across both on-premises and cloud, or even across multiple cloud environments. Especially in the current pandemic-hit business landscape, data virtualisation provides organisations with the agility and flexibility to take a data management approach that best suit their business requirements, and not be restricted by concerns over security or data management.
Data Virtualisation in Action
To get a sense of how data virtualisation works in the real world, consider the case of a leading global resources company. Over time, this company had spread its enterprise data across multiple continents, across disparate on-premises systems and multiple cloud sources.
The company’s data teams had been running daily, weekly, and monthly reports for a wide variety of different divisions, such as mining, oil and gas, human resources, finance, and health and safety. But when analysts needed to access data from multiple sources, the team had to first copy data into a new physical location. This cost the company crucial reporting time and resources. The company was also running advanced analytics projects in AWS, but for many use cases, the company needed to merge this data with the data in several on-premises sources.
The company implemented an enterprise-wide data virtualisation layer across its four major facilities around the globe, embracing all data sources, and this layer seamlessly integrated the data between the multiple cloud systems and the on-premises systems. Data virtualisation dramatically accelerated access to dashboards, fundamentally enabling self-service access to business users. The company established the data virtualisation layer as the universal data access point for all enterprise data, making a much wider variety of enterprise data available for business reporting and advanced analytics, moving forward.
Data virtualisation has been evolving for many years, and many companies are beginning to leverage it to seamlessly integrate data between multiple clouds, enabling companies to gain the best of multiple worlds.