Sr Developer – ETL Pyspark

Job title:

Sr Developer – ETL Pyspark

Company

Randstad

Job description

This Senior Consultant/Developer will work on creating, operating (run processes) and maintaining the portfolio of products assigned to them as well as, test, prepare deployment packages, deploy
code/script changes, support user tests and inquiries and perform or follow up on application administration tasks required for the maintenance of the product.Advantages
Competitive Compensation
Diverse Projects
Clear Expectations & Structure
Possibility of ExtensionResponsibilities
The main deliverables are:

• Understand systems and Business processes related to the products in their assigned

portfolio

• Works closely with analytical teams to develop, maintain, and support products using
• Python and/or R and Spark programming languages.
• Provide support and guidance on creating top-quality clean code and participate in code review.
• Analyses Software deficiencies/problems and support to fix them
• Triages analysts’ issues to proper administrators if required.
• Prepares documentation, reports, and presentations related to projects and activities.
• Shares knowledge and expertise across the organization; Provides learning opportunities

and knowledge sharing to staff related to advanced analytics, machine learning

• methods and data visualization.
• Process operational tasks in these products like running scripts in test and prod

environments, change config files, etc, to support scheduled releases of the products

• Perform maintenance tasks in these products like changes to roll-over to next Fiscal Year

and others

• Develop code changes as per approved product updates requirements to these products
• Test thoroughly the changes and maintain test scenarios as applicable
• Perform deployment to test environments and support QA team (as applicable) and users

Testing

• Support user’s inquiries
• Communicate well with Business users as well as other groups to ensure all tasks required
• for the products maintenance are covered
• Perform or follow up with application administration tasks as required
• Prepare deployment packages and required release tickets to deploy the changes
• Follow up on deployment tasks with all groups involved
• Create or modify documentation related to the changes in the product’s technical design,
• ETL mappings and others as advised by ithe nternal IT team.

The successful candidate must have:

• Profound knowledge and experience in developing in PySpark data pipelines
• Experience with AWS EMR clusters
• High knowledge in shell scripting.
• Experience with Apache Airflow
• Proven ability to effectively manage multiple deliverables on time and first time right
• Ability to handle heavy workload
• Willingness to work on what is required for the products maintenance, being it operations,
• development or other tasks
• Strong trouble shooting and problem solving skills with logical and good attitude team player
• with strong oral and interpersonal skills
• Excellent communication skills
• Ability to communicate to different audience, being Business users or managers as well as
• technical peers

Qualifications
Skills Required

• Python and Spark programing languages to manipulate data and draw insights from large datasets 3+ Years of experience
• Experience with AWS EMR platform 2+ Years of experience required
• Hands-on programming skills, able to write modular, maintainable code
• preferably Python &; SQL. Good understanding of SQL, dimensional modeling, and analytical big data warehouses. 5 + Years of experience
• Working knowledge of Visual Studio, Jupyter notebook, and Git 3+ Years of experience
• Experience with Apache Airflow 1+ Years of experience
• Experience with Python programming in the context of data processing / data science applications 2+ Years of experience
• Profound knowledge and experience with Oracle database 5 + Years of experience
• Profound knowledge and experience in Oracle PL/SQL 5 + Years of experience
• Profound knowledge and experience in shell scripting in Linux/Unix Environment. 5 + Years of experience
• Experience in creating and maintaining documentation of project ETL design, technical artifacts and ETL environment. 5 + Years of experience
• Experience developing in an AWS Cloud environment 2+ Years of experience
• Working experience with R is an asset N/A Na

Summary
The Senior Consultant/Developer will be responsible for developing, managing, and maintaining the assigned portfolio of products. This includes testing, preparing deployment packages, deploying code/script updates, supporting user testing and inquiries, and handling or following up on necessary application administration tasks for ongoing product maintenance.Randstad Canada is committed to fostering a workforce reflective of all peoples of Canada. As a result, we are committed to developing and implementing strategies to increase the equity, diversity and inclusion within the workplace by examining our internal policies, practices, and systems throughout the entire lifecycle of our workforce, including its recruitment, retention and advancement for all employees. In addition to our deep commitment to respecting human rights, we are dedicated to positive actions to affect change to ensure everyone has full participation in the workforce free from any barriers, systemic or otherwise, especially equity-seeking groups who are usually underrepresented in Canada’s workforce, including those who identify as women or non-binary/gender non-conforming; Indigenous or Aboriginal Peoples; persons with disabilities (visible or invisible) and; members of visible minorities, racialized groups and the LGBTQ2+ community.Randstad Canada is committed to creating and maintaining an inclusive and accessible workplace for all its candidates and employees by supporting their accessibility and accommodation needs throughout the employment lifecycle. We ask that all job applications please identify any accommodation requirements by sending an email to accessibility@randstad.ca to ensure their ability to fully participate in the interview process.

Expected salary

Location

North York, ON

Job date

Sun, 23 Feb 2025 01:00:51 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (jobsnearcanada.com) you saw this job posting.