Why consider a Modern Data Platform on Azure Cloud?
Many organizations are moving their on-prem IT infrastructure and functions to cloud as part of their digital transformation strategy. There are a number of reasons cloud is a central component of digital transformation, such as operationalization of fixed cost, reduced overall cost, and faster time-to-market. Cloud infrastructure also offers unmatched flexibility and scalability, and is a reliable and robust component of a good business continuity strategy. Additionally, cloud providers offer a baseline of security and governance controls, and provide a variety of “as a Service” options including infrastructure, platform, and software that further help accelerate time-to-market.
Characteristics of a Modern Data Platform
Data processing needs are constantly evolving, with the need for faster intake and analytics of structured, semi-structured, and unstructured data changing every day. A Modern Data Platform enables the storage, processing, and consumption of large volumes of complex data, exhibiting the following characteristics:
- A single, powerful, and extensible data platform
- Ability to handle a broad array of data types
- Ability to support new types of analytics
- Automated and robust data pipelines and ETL processes
- Secured access of data for a variety of users – analysts, data engineers, and data scientists
- An easy-to-use environment for developing data applications, including several options for self-service
- Guaranteed and cost-efficient scale, performance, and concurrency
- Requiring the least maintenance, so that users can focus on delivering business objectives
Why Azure
At a high level, a Modern Data Platform on Azure provides a single view of enterprise data, utilizing the following:
- Centralized storage in Data Lake
- Several automated and powerful tools such as Azure Data Factory (ADF)
- A Spark-based Databricks environment for transformation
- A Synapse Analytics data warehouse
- Azure Event Hub
- Azure Stream Analytics
- Azure Analytics Services
- Containers
- Kubernetes services
- Integration of GitHub and nearly all open source Apache data products
The Modern Data Platform uses Gamma architecture to support both batch and stream processing and consumptions. The Azure Cosmos DB provides capability for storing NoSQL data for consumption by APIs in real time.
Benefits of a Modern Data Platform on Azure
Low learning curve – Microsoft’s SQL Server database and Power BI are extensively used in the industry. Synapse – Azure’s data warehouse product – is based on SQL Server Warehouse and Power BI is offered on Azure, offering familiarity across data and analytics use cases. Many Azure software components are also similar or identical to their on-prem counterparts.
Minimal migration cost – Azure provides several tools to help with migration, including Azure Migrate, Azure Site Recovery, Azure Database Migration Service, and the Azure Data Box.
Minimal application downtime – the construction of a Modern Data Platform on Azure can be accomplished without impacting applications running on-prem. Azure Data Factory is used to bulk load data from on-prem data sources and to schedule workflows for incremental loads.
Security and Compliance – Azure provides a variety of services for securing the environment and addressing compliance needs, including Azure Security Center, Azure Blueprints, Azure Policy, Azure Key Vault, Azure Information Protection, and Azure Multi-Factor Authentication.
