Data Engineer

Overview

Develop and maintain AWS-based data pipelines and AI/ML tools for social housing insights.

Key Responsibilities

• ai engineering
• etl pipelines
• solr search
• data ingestion
• mlops
• client liaison

Tasks

Ingesting and understanding new data sources, evolving our data approach (e.g., to a fully-fledged data fabric), implementing and streamline MLOps processes and exploring how we can sensibly and safely utilise artificial intelligence (AI) to turbo charge our work (e.g., smart data quality monitoring, meta data curation and service optimisation). As part of this you may also be asked to liaise with our clients (directly or through events, conferences, webinars etc.), to help support their data driven journeys. -Utilising AI within data engineering to drive performance -Balances delivery detail with big-picture thinking -Building scalable ETL/ELT pipelines with Python -Facilitating search tools such as Solr

Requirements

• aws
• apache airflow
• python
• dbt
• terraform
• sql

What You Bring

Please note - we are not expecting experience in all of the skills below .. we do invest in professional training and development for our employees and we have some very experienced Data Engineers here that you can learn from Skills required: Ideally 3 years experience as a Data Engineer, AWS, a keen interest in AI & ML -Ability to work with and develop APIs (including data transformations). -Understanding of Cloud security frameworks (specifically on the AWS Cloud) including appropriate data encryption -Apache Airflow (DAG design, scheduling, monitoring, scaling) -Can influence and collaborate across teams -Tools and methods to develop comprehensive data reliability and active metadata solutions. -Is proactive, self-motivated, and solutions-focused -Strong data reliability/observability, data governance and information security credentials. -Data quality checks, monitoring, and observability -Amazon Redshift (query tuning, distribution/sort keys, workload management) -Full data engineering cycle knowledge (tools and skills) for stream data -Ability to work effectively both as an individual (e.g., during remote deep work) and within a team - supportive, collaborative. -Python (data processing, automation, testing) -AWS Lambda (Python-based serverless pipelines, event-driven processing) -Exposure to data governance, cataloguing, and lineage tools -Ability to work with a range of structured, semi-structured and unstructured file formats including Parquet, json, csv, xml, pdf, jpg. -Enjoys mentoring and helping others grow -Ideally a degree in Information Systems, Computer Science, Information Technology, Software Engineering or similarly related and quantitative discipline. -AWS .. has used AWS commercially -Clear written and verbal communication, including effective communication with non-technical audiences. Business acumen, i.e., understanding the business context with a proven ability to align and actively support business goals and OKRs. -MLOps experience including familiarity with tools such as DVC & mlflow -Best practices for dependency management, retries, and alerting -Data modelling (normalisation, dimensional) -dbt (modeling, testing, documentation, deployment) -Circa 3 years of commercial experience working primarily in an AWS Cloud environment using approaches/tooling like ours (see technical skills), delivering scalable, performant, reliable solutions. -Experience with logging/monitoring -SQL (advanced query writing and optimization) -Infrastructure as Code (Terraform etc.) -Ability to deliver data deidentification and anonymisation solutions. -An individual who takes a keen interest in data engineering's frontiers and best practice. Someone who is a critical thinker and a problem solver. -CI/CD for data pipelines (Git, GitHub Actions, etc.)

People Also Searched For

Benefits

Competitive salary and rewards package including: - Private Health care , 4 x salary Life cover, 25 days annual leave, increasing to 28 after 3 year’s service,salary sacrifice pension scheme and much more .. Salary: Competitive plus excellent benefits In short there will be plenty to keep you interested, your technical and power skills developing and a real chance to drive innovation and change. Alongside the work opportunities as described your development will be supported, you will be sensibly renumerated, we will provide a compelling benefits package, and we are a great bunch of folks to work with - though we would say that!

About Mobysoft

-Revolutionized housing management with its predictive analytics software. -Offers a dynamic approach to understanding tenant behavior and optimizing financial management. -Enables better tenant relationships and financial forecasting for housing providers. -Uses data-driven insights to provide predictive solutions. -Integrates seamlessly with existing housing management systems. -Helps housing providers make informed, proactive decisions that boost efficiency.

Sector Specialisms