Data Engineer II

Key Responsibilities

• data quality
• etl design
• data pipeline
• warehouse design
• performance optimization
• code review

Tasks

-Implement data quality checks and validations within data pipelines to ensure accuracy, consistency and completeness of data. -Collaborate with cross-functional teams to promote AI solutions in the enterprise. -Profile source system data and design data warehouse and lake schemas -Optimize data pipelines for performance, scalability and efficiency. -Design, implement, and support ETL, ELT and integration pipelines -Apply best practices in solution development, quality control and security in implementing pipelines. -Research, analyze, recommend, and select technical approaches for solving challenging and complex development and integration problems -Participate in code and design review to ensure alignment to standards and best practices -Transform raw data into usable format based on analytics, reporting and integration requirements.

Requirements

• azure
• python
• power bi
• nosql
• t-sql
• agile scrum

What You Bring

-Experience with Agile/Scrum methodology preferred -Experience with master data management -Experience developing integration solutions using a low-code platform such as Boomi, Snaplogic or Mulesoft. -Excellent analytical, conceptual, and problem-solving abilities with aptitude for acquiring new technical skills -Highly motivated team player with strong interpersonal skills -Experience with NoSQL database systems and building data pipelines to ingest unstructured and streaming data -5+ years of work experience in data management disciplines, including data modeling, integration and development of ETL/ELT pipelines. -Experience developing and promoting work through devops/dataops pipelines and utilizing source control -Experience preparing data for Data Science and Machine Learning use cases -Experience with T-SQL (queries, stored procedures, and DDL) -5+ years of data solution delivery experience, preferably in the Microsoft Azure platforms and tools (Azure Data Factory, Azure SQL, Azure Data Lake Gen2, Azure DevOps). Experience in Microsoft Fabric would be a plus. -Experience querying API endpoints -Solid understanding of data warehouse and data lake concepts and design (including star schemas) -Experience with Power Query -Experience developing solutions in a Lakehouse platform -Experience designing reports using Power BI, Power Query, and DAX -Python proficiency for data management, ML and AI development.

People Also Searched For