The client, India's first publicly listed retail real estate investment trust, manages a vast portfolio of 17 Grade A urban consumption centers, two hotel assets, and three office assets, covering approximately 9.9 million square feet of gross leasable area across 14 cities. They faced significant challenges managing their expansive portfolio due to scattered data across various departments and vendors. This lack of a centralized data management system hindered their ability to perform business intelligence (BI) and predictive analysis, limiting the data's potential use for marketing, sales, and brand partnerships.
The Challenges
The client's data was dispersed across multiple sources and formats, managed by different external vendors. The reliance on these vendors for data access created bottlenecks, causing inefficiencies in business processes. Additionally, the data was unstandardized, making it difficult to perform any significant BI analysis or leverage insights for marketing and operational improvements. The lack of cohesive data visualization and reporting capabilities also meant that the client was unable to share valuable data-driven insights with various brands housed in its malls, missing opportunities to drive performance and strategy
Case Highlights
- Centralized data storage using Azure SQL
- Standardization of data across all formats and sources.
- Elimination of vendor dependency
- Improved reporting that led to better decision-making.
- Scalable, automated pipelines future-proofing the architecture.
- Secure and compliant data management
Nexdigm was tasked with addressing these challenges by developing a solution that would centralize data and eliminate dependency on external vendors. The goal was to create a Single Source of Truth that would allow the client to standardize, store, and transform its data into actionable insights, enabling enhanced reporting and analytics.
The Solution
After a cost-benefit analysis, Azure was identified as the most suitable platform for the data transformation and storage solution. Nexdigm designed and implemented a comprehensive data architecture, including the development of automated Azure cloud pipelines within Azure Data Factory (ADF) and Azure Databricks for data extraction, while Azure SQL served as the central data warehouse.
The solution architecture was designed as follows:
- Data Ingestion: Data from multiple sources, both manual and API-based, was ingested and stored in a centralized SharePoint folder.
- Automated Data Movement: Power Automate was deployed to transfer raw data to Azure Data Lake Storage (ADLS), reducing manual intervention and ensuring consistent data flow.
- Bronze Layer: The raw data was securely stored in Excel/CSV formats within ADLS, where Databricks performed cleaning and harmonization tasks to prepare the data for further transformations.
- Silver Layer: In this layer, Databricks applied advanced filtering and merging operations to the data, converting it into Parquet format for optimized storage and performance.
- Gold Layer: Structured and transformed data was stored in Azure SQL, ready for use in Power BI and Tableau for reporting and analytics purposes. This allowed the client to create detailed tables and views for advanced decision-making.
- Security & Authentication: Nexdigm ensured top-notch security using Azure Key Vault to manage access authentication and Azure Active Directory to enforce role-based access control, ensuring that sensitive data was securely handled.
- Orchestration & Scalability: The use of Azure Data Factory Pipeline enabled smooth data orchestration, ensuring the architecture could scale with future growth and data needs. This automated process eliminated manual oversight, ensuring continuous data processing and availability.
The Impact
The implementation of a centralized data warehouse significantly reduced dependency on external vendors for data access. With all data integrated into Power BI, the performance of dashboards improved drastically, enabling timely and more effective business decisions. This newfound agility allowed the client to generate real-time insights, providing valuable information to both brands and investors
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
