Data Engineer
Job code:
DM-21374
Location:
ACT
Department:
Australian Electoral Commission
Application closing date:
11 December 2023
Start date:
1 January 2024
Contract duration:
12 months
Number of extensions:
1
Requirements
About our requirements The role will be responsible for, but not limited to: • Collaborate with team members and stakeholders to design data integration solutions that meet business requirements, utilising Microsoft Azure tools such as Azure Data Factory and SQL. • Develop, implement, and maintain ETL pipelines to ensure efficient and secure data transfer between various systems, adhering to best practices for data processing and transformation. • Design and maintain data models, ensuring data accuracy, consistency, and availability while considering scalability and performance optimisations. • Implement data validation and quality checks to identify and rectify data anomalies, errors, and inconsistencies within integrated systems. • Continuously monitor and optimise data integration processes to enhance performance, reduce latency, and minimise resource consumption. • Create and maintain comprehensive documentation for data integration solutions, including technical specifications, data flow diagrams, and operational procedures. • Work closely with cross-functional teams, including developers, and system analysts, to understand data requirements and ensure integrations align with business objectives. • Troubleshoot and resolve data integration issues, working collaboratively with the team to implement effective solutions and prevent recurrence. • Stay updated on emerging technologies and industry trends related to data engineering and integration, applying this knowledge to improve integration processes and tools.
About the Project
The Enterprise Integration Team is responsible for designing, building, and supporting integrations between on-premises and cloud-based enterprise applications, utilising Microsoft Azure tools such as Azure APIM, Azure Service Bus and Azure Data Factory. The team provides ongoing support, performance monitoring and troubleshooting, while also collaborating with other software development teams to design and build APIs and data pipelines supporting multiple modernisation projects. Additionally, the team works with Master Data Management (MDM) and Identity and Access Management (IDAM) to ensure data integrity and security, streamlining data flow and enhancing productivity across the agency.
Mandatory: • Proven experience in data engineering, including data integration, ETL processes, and data modelling. • Strong SQL skills and experience with data warehousing concepts. • Knowledge of data governance, security, and compliance best practices. • Excellent problem-solving skills and the ability to work effectively in a collaborative team environment. • Effective communication skills for collaborating with technical and non-technical stakeholders. • Demonstrated ability to build and maintain effective working relationships. • Ability to work independently and be a self-motivated individual.
Desirable: • Proficiency in Microsoft Azure tools and services, such as Azure Data Factory, and Azure SQL Database. • Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience). • Relevant certifications (e.g., Microsoft Azure Data Engineer, AWS Certified Data Analytics) are advantageous.