Azure Data Solutions Architect

About the position

Job Title: Technical Architect / Azure Data Solutions Architect Location: Petaling Jaya, Malaysia (Hybrid) Role Type: Permanent Responsibilities: • Lead the architecture, design, and implementation of advanced analytics solutions using Azure Databricks and Microsoft Fabric. • Develop scalable data solutions with a strong focus on big data technologies, data engineering, cloud computing, and SQL. • Collaborate closely with business stakeholders and IT teams to gather requirements and deliver effective solutions. • Oversee end-to-end implementation of data solutions, ensuring alignment with business needs and best practices. • Lead the development of data pipelines and ETL processes using Azure Databricks, PySpark, and related tools. • Integrate Azure Databricks with Azure services (Azure Data Lake, Azure Synapse, Azure Data Factory) and on-premise systems. • Provide technical leadership and mentor data engineering teams, fostering continuous learning and improvement. • Ensure proper documentation of architecture, processes, and data flows, adhering to security and governance standards. • Enforce best practices in code quality, data security, scalability, and performance. • Stay up to date with the latest developments in Databricks and related technologies to drive innovation. Essential Skills: • Strong experience with Azure Databricks, including cluster management, notebook development, and Delta Lake. • Proficiency in big data technologies (e.g., Hadoop, Spark) and data processing frameworks (e.g., PySpark). • Deep understanding of Azure services such as Azure Data Lake, Azure Synapse, and Azure Data Factory. • Experience with ETL/ELT processes, data warehousing, and building data lakes. • Strong SQL skills and familiarity with NoSQL databases. • Experience with CI/CD pipelines and version control systems (e.g., Git). • Knowledge of cloud security best practices. Soft Skills: • Excellent communication skills with the ability to explain complex technical concepts to non-technical stakeholders. • Strong problem-solving abilities with a proactive approach to issue resolution. • Proven leadership skills with the ability to manage and mentor data engineering teams. Nice-to-Have Skills: • Experience with Power BI for dashboarding and reporting. • Familiarity with Microsoft Fabric for analytics and integration. • Experience with Spark Streaming for real-time data processing. • Knowledge of Azure Resource Manager (ARM) templates and Infrastructure as Code (IaC) practices. Experience: • 12+ years of experience in developing data ingestion and transformation pipelines using Databricks, Synapse notebooks, and Azure Data Factory. • Hands-on experience with Delta tables, Delta Lake, and Azure Data Lake Storage Gen2. • Experience using Auto Loader and Delta Live Tables for efficient data ingestion and transformation. • Proficiency in building and optimizing query layers using Databricks SQL. • Experience integrating Databricks with Azure Synapse, ADLS Gen2, and Power BI for end-to-end analytics solutions. • Proven experience in developing, optimizing, and deploying Power BI reports. • Familiarity with modern CI/CD practices in Databricks and cloud-native environments. Recruitment Partner: Sperton: This position is exclusively managed by Sperton, a global talent partner connecting high-performing professionals with leading organizations worldwide.

Place of work