Data & AI Platforms Engineer

Overview

Administer and optimise Databricks on AWS, managing IAM, clusters, security and integration.

Key Responsibilities

• cluster management
• data governance
• iam management
• network security
• issue resolution
• cloud integration

Tasks

-Administer, oversee, and monitor Databricks clusters on AWS, maintaining optimal performance and functionality. -Work in partnership with data governance teams to promote and implement best practices in data management. -Configure and manage Identity and Access Management (IAM) within the Databricks environment to ensure secure and appropriate access control. -Support the implementation of robust network security measures, including VPC configurations and firewall rules, to safeguard the platform. -Partner with AWS and Databricks support teams to resolve complex technical issues and enhance system reliability. -Collaborate closely with cloud architects, data engineers, and security teams to facilitate seamless integration with AWS services.

Requirements

• python
• sql
• gitlab
• aws
• databricks
• bachelor's

What You Bring

-Experience in the utilities sector (desirable). -Proficiency in scripting languages (e.g., Python, SQL, Bash) for automation. -Familiarity with DevOps Tools (e.g., Gitlab) -Familiarity with AWS services (e.g., S3, DMS, EC2, IAM, Lambda, Glue) and their integration with Databricks. -Strong experience with Databricks on AWS, including cluster management, performance tuning, security, and data entitlements management. -Strong experience in Databricks administration. -Bachelor’s degree in computer science or a related field.

People Also Searched For

About Apa Group

-It builds and runs major gas pipelines – Moomba–Sydney, Roma–Brisbane, SEAGas among others – stitching regions together. -Its electricity arm links grids via Murraylink, Basslink and Directlink, powering cross‑boundary interconnections. -Wind‑farm holdings like Emu Downs and Hallett, plus gas‑fired stations in Mount Isa, show its generation edge. -Projects range from interstate pipeline networks to high‑voltage interconnectors and hybrid generation sites.

Sector Specialisms

Security Clearance

-candidates will undergo background vetting including police checks.