Cloud Data Engineering Manager

Description

• Provide technical mentorship and guidance to junior engineers, fostering a culture of innovation and continuous learning.
• Communicates with Stakeholders.
• Lead the development of data pipelines and ETL/ELT workflows using Azure Data Factory, ensuring efficiency, reliability, and reusability across projects.
• Manage CI/CD processes for data infrastructure using Azure DevOps, ARM templates, and YAML pipelines. Monitor and optimize data workloads using Azure Monitor, Log Analytics, and SQL Profiler, ensuring high availability, performance, and cost-efficiency.
• Implement robust data ingestion frameworks for both structured and unstructured data sources, including APIs, on-premises databases, IoT, and 3rd party systems like Workday and Procore.
• Implement data validation and quality checks to ensure accuracy and consistency.
• Accepts tasks from management and the client.
• Build real-time and batch data processing solutions leveraging Azure Stream Analytics, Event Hubs, and Databricks, where applicable.
• Design, build, and maintain scalable and secure cloud-based data solutions primarily within the Microsoft Azure ecosystem.
• Work independently and with the business stakeholders to understand their requirements and drive business outcomes.
• Design and enforce data governance standards, metadata management, and data security policies, including row-level security and data masking using Azure Purview and SQL Server security roles.
• Collaborate with data scientists, analysts, and business stakeholders to design and deliver semantic models, Power BI datasets, and dashboards that translate raw data into actionable insights.
• Data Pipeline Development: Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data. Develop robust ETL (Extract, Transform, Load) processes to integrate data from diverse sources into our data ecosystem.
• Develop and optimize data lake and data warehouse architectures using Azure Data Lake Storage Gen2, Azure Synapse Analytics, and SQL Server to support advanced analytics and business intelligence initiatives.

Requirements

• Familiarity with Agile/Scrum methodologies, source control via Git, and project management tools like Azure Boards or Jira.
• Hands-on experience integrating with Microsoft Power Platform, especially Power BI, including DAX, M language, and dataset optimization.
• Ability to think strategically about design and architectural aspects of the data infrastructure.
• Bachelor’s or Master’s degree in Computer Science, Information Systems, Data Engineering, or related field.
• Knowledge of indexing and query optimization techniques.
• Experienced in SQL Server Database Administration, the utilization of SQL Server database for backend development and integration services, and possessing advanced SQL concepts.
• Experience in implementing data modeling best practices (star/snowflake schema), dimensional modeling, and snapshotting techniques.
• Experience using aggregate functions.
• Experience with data governance and compliance, including data lineage, audit trails, sensitive data classification, and GDPR/CCPA requirements using Azure Purview or similar tools.
• Familiarity with referential integrity best practices.
• Strong expertise in T-SQL, including writing optimized queries, stored procedures, and views for high-performance analytics.
• Proven experience in building enterprise-grade solutions on Azure, including: Azure Data Factory for orchestration, Azure Synapse and SQL Server for data warehousing, Azure Data Lake Storage Gen2 for scalable storage, and Azure Functions and Logic Apps for automation.
• Solid understanding of data concepts, including ODS, data warehousing, and data storage.
• Proficiency with joins and subqueries.
• Understanding of database normalization principles.
• 7 years of professional experience in data engineering, with at least 3 years focused on Microsoft cloud data technologies.

Benefits

Training + Development

Interview process

Visa Sponsorship

Security clearance