Data Engineer - Remote, Anywhere in Canada
EDC is Canada’s export credit agency, offering information, knowledge, and innovative commercial solutions to help Canadian exporters and investors expand their international business. We take on risk so Canadian businesses can take on the world.
When you join our team, you’ll be helping Canadian companies bring their vision, passion and innovation to the global marketplace. Your knowledge and expertise will support more than 25,000 Canadian companies and their customers in as many as 200 markets worldwide. You’ll work shoulder-to-shoulder with the best and brightest in an inclusive, collaborative environment that fosters professional development and success. And you’ll know that you’re making a difference every day—for companies, for Canada and for the people you work with.
Year after year, EDC has been recognized as a Top 100 and Top Family-Friendly Employer.
Team & Job Overview
The Data Engineering Services team is seeking an individual that has experience in delivering, implementing, and collaborating with stakeholders to achieve the best results for EDC against its Data & Analytics strategy.
The Data Engineer will be responsible for delivering data through data pipelines on EDC’s data platform to meet our stakeholder’s needs. This role will be joining the team that is responsible for building out the Data Engineering practice at EDC using the latest in modern data platform technologies.
- Designs, develops, assesses, modifies, adapts, integrates and implements short- and long-term solutions to information technology (IT) needs through new and existing applications, web applications, databases, systems infrastructure, network systems and applications infrastructure;
- Reviews system requirements and business processes; codes, tests, debugs and implements and sustains software solutions and systems;
- Conducts a wide range of quality control tests and analyses to ensure that systems meet or exceed specified standards and end-user requirements;
- Ensures that system improvements and systems tests are successfully completed and documented, all problems are resolved, and systems are successfully implemented;
- Responsibilities are within the IT Development Discipline as a generalist or in a combination of disciplines.
- Plans, coordinates and oversees activities related to the design, development and integration of information system, operations systems and/or datamarts;
- Review and analyze the effectiveness and efficiency of existing systems and develop strategies for improving or further leveraging these systems;
- Reviews systems models, specifications, diagrams and charts to provide direction to system programmers and manages third party vendor (managed services) relationships;
- Develops standards and processes for coding, deployment, testing, and governance;
- Leads the provision of expertise for the complex elevated technical support issues including solution lifecycle management.
- Undergraduate degree in Computer Science, Management Information Systems, or in a related field;
- Minimum 5 years’ experience in IT systems development or related experience;
- Advanced knowledge of IT systems development principles and practices;
- Minimum 3 years’ experience in the designing and building data pipelines following software engineering concepts and best practices.
Skills and Knowledge
- Proven design, build and implementation of batch and real-time data pipelines. Driven by automated repeatable delivery of data that aligns to enterprise data governance standards
- Experienced collaborator working with various stakeholders across the business, data scientists and IT. Working closely building relationships, refining data requirements to meet various data and analytics initiatives and data consumption requirements
- Experience in developing and proposing data models that conform to requirements. Responsible to ensure the proposed design, optimally addresses access and query patterns; data consumption and adheres to internal architecture standards
- Experience working with modern data processing technology stacks: Spark, Hive
- Experience working with streaming data architecture and technologies for real-time: Spark Streaming, Kafka, Flink, Storm
- Experience working with relational and non-relational database technologies: SQL Server, Synapse Oracle, Cassandra, MongoDB, CosmosDB, HBase
- Experience working with cloud data platform technologies: Azure Data Lake Store, Azure Databricks, Azure Synapse and Azure EventHub
- Programming experience in Spark using modern languages such as Scala, Java and/or Python
- Experience working with source code and configuration management environments such as Azure DevOps, Git, Maven, Nexus
- Understanding of agile development methods including core values, guiding principles, and key agile practices
- Understanding of the theory and application of Continuous Integration/Delivery
- Experience with Azure Databricks and Delta Lake
- Experience with Scala and Spark
- Bilingual in both official languages (French and English)
- For this position, EDC is recruiting top talent across Canada. We offer full flexibility to work remotely during and after the pandemic.
- $64,100 - $96,100, plus performance-based incentive
Please note that EDC will be following a mandatory vaccination policy.
How to apply
Only candidates selected for an interview will be contacted.
Application deadline on www.edc.ca/careers : Jan 28, 2022, 11:59:00 PM EDC is committed to employment equity and actively encourages applications from women, Aboriginal people, persons with disabilities and visible minorities. If selected for an interview, please advise us if you require special accommodation. Candidates must meet the requisite government security screening requirements.