Senior Data Engineer

As part of this you may also be asked to liaise with our clients (directly or through events, conferences, webinars etc.), to help support their data driven journeys.

Ingesting and understanding new data sources, evolving our data approach (e.g., to a fully-fledged data fabric), implementing and streamline MLOps processes and exploring how we can sensibly and safely utilise artificial intelligence (AI) to turbo charge our work (e.g., smart data quality monitoring, meta data curation and service optimisation).

• Building scalable ETL/ELT pipelines with Python
• Balances delivery detail with big-picture thinking
• Facilitating search tools such as Solr
• Utilising AI within data engineering to drive performance

Ideal skills: Data Engineer, MLOps, AI, Smart Data, AWS Certified, Data Governance, Data Transformation, Data Modelling, ELT.

• Ability to work effectively both as an individual (e.g., during remote deep work) and within a team - supportive, collaborative.
• Apache Airflow (DAG design, scheduling, monitoring, scaling)
• Is proactive, self-motivated, and solutions-focused
• Infrastructure as Code (Terraform etc.)
• AWS certification - AWS Certified Data Engineer, Associate certification.
• Ability to work with a range of structured, semi-structured and unstructured file formats including Parquet, json, csv, xml, pdf, jpg.
• SQL (advanced query writing and optimization)
• Can influence and collaborate across teams
• An individual who takes a keen interest in data engineering's frontiers and best practice. Someone who is a critical thinker and a problem solver.
• AWS Lambda (Python-based serverless pipelines, event-driven processing)
• 4-5+ years of commercial experience working primarily in an AWS Cloud environment using approaches/tooling like ours (see technical skills), delivering scalable, performant, reliable solutions.
• Python (data processing, automation, testing)
• MLOps experience including familiarity with tools such as DVC & mlflow
• Understanding of Cloud security frameworks (specifically on the AWS Cloud) including appropriate data encryption
• Amazon Redshift (query tuning, distribution/sort keys, workload management)
• Ability to deliver data deidentification and anonymisation solutions.
• Enjoys mentoring and helping others grow
• Exposure to data governance, cataloguing, and lineage tools
• CI/CD for data pipelines (Git, GitHub Actions, etc.)
• Strong data reliability/observability, data governance and information security credentials.
• Full data engineering cycle knowledge (tools and skills) for stream data
• Tools and methods to develop comprehensive data reliability and active metadata solutions.
• Experience with logging/monitoring
• Ability to work with and develop APIs (including data transformations).
• Data quality checks, monitoring, and observability
• Best practices for dependency management, retries, and alerting
• Data modelling (normalisation, dimensional)
• Clear written and verbal communication, including effective communication with non-technical audiences. Business acumen, i.e., understanding the business context with a proven ability to align and actively support business goals and OKRs.
• An honours degree in Information Systems, Computer Science, Information Technology, Software Engineering or similarly related and quantitative discipline.
• dbt (modeling, testing, documentation, deployment)

Competitive salary and rewards package including: - Private Health care , 4 x salary Life cover, 25 days annual leave, increasing to 28 after 3 year’s service,salary sacrifice pension scheme and much more ..

Salary: Competitive plus excellent benefits

Alongside the work opportunities as described your development will be supported, you will be sensibly renumerated, we will provide a compelling benefits package, and we are a great bunch of folks to work with - though we would say that!

In short there will be plenty to keep you interested, your technical and power skills developing and a real chance to drive innovation and change.