Data Engineers

Job code:

DM-20886

Location:

ACT

Department:

Department of Agriculture, Fisheries and Forestry

Application closing date:

7 November 2023

Start date:

20 November 2023

Contract duration:

12 months

Number of extensions:

Requirements

The Department of Agriculture, Fisheries and Forestry (DAFF) is looking for Data Engineers with strong experience developing Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes across several data and analytics platforms to join its Enterprise Analytics section.

The role will be responsible for design, development and unit testing activities across several data movement and transformation processes within DAFF. These data movement processes are being developed to focus on the preparation of data for use in decision making processes across the department, utilising modern cloud technology (Azure) to enable operational analytics use cases.

The successful candidate will require experience with the following techniques and technologies:

MS Azure StackData IntegrationData FactorySQL Server Integration ServicesDatabricksData StoreSQL ServerData Lake StorageAnalyticsAzure DatabricksAzure Machine LearningArcGIS EnterpriseDevelopment Tools – DevOps, Visual Studio, VS CodeData technology solutions – sourcing (Oracle, Ingres, Azure, SQL Server), collecting, ingesting and storingData PreparationTransformation of data into formats tailored to analytics use casesParquetDelta

Estimated start dateMonday, 20 November 2023

Initial contract duration7 Months (until 30 June 2024)

Extension term12 months

Number of extensions2

Location of workACT, Offsite

Working arrangementsPreference is for a minimum of 2 days a week onsite. Working away from the office arrangements are subject to a formalised assessment and pattern that is amendable to operations of the team.

Security clearanceMust be able to obtain Baseline

Criteria

Essential criteriaWeighting

1. Experience developing ETL/ELT processes for data movement and transformation.25%

2. Experience preparing data optimised for query performance in cloud computed engines. E.g. • Distributed computing engines (Spark) • Graph Databases • Azure SQL25%

3. Experience working with Engineering, Storage and Analytics services in cloud infrastructure.25%

4. Working with multi-disciplinary teams using an Agile methodology25%

Desirable criteria1. Experience working with Azure Data Factory and Databricks2. Experience/Knowledge of working with Data Lake and Lakehouses