Permanent
Senior Data Engineer
Unknown, Unknown
unknown
About this role
Employer: Goldman Tech Resourcing
Our client is seeking a Senior Cloud Data Engineer to join their Engineering team. In this role, you will be a key technical contributor responsible for building, optimizing, and maintaining high-performance data processing engines. You will apply your deep expertise in Spark Best Practices to develop modular, portable PySpark applications, ensuring their platform remains provider-agnostic while operating within modern environments like Microsoft Fabric. As an experienced member of the Data Engineering team, you will focus on refactoring legacy logic into scalable Python-centric solutions, optimizing their data lakehouse architecture, and contributing to the overall technical excellence of our client's data platform
Requirements:
· Bachelor’s degree in computer science, Information Systems, Engineering, or a closely related field.
· Senior-Level Spark: 6+ years of experience with Spark/PySpark, with the ability to diagnose performance bottlenecks via the Spark UI and optimize complex DAGs.
· Advanced Python: Strong proficiency in production-grade Python, with experience building reusable libraries and implementing automated testing.
· SQL Proficiency: Solid T-SQL skills to accurately interpret and migrate existing logic into the new Python-centric environment.
· Azure Synapse Analytics & Dedicated SQL Pools: Solid experience working within Azure Synapse, specifically with Dedicate SQL Pools and Data Factory working with complex pipelines
· Data Lakehouse Expertise: Hands-on experience with Delta Lake and Parquet, specifically in high-volume environments.
· Agnostic Mindset: Experience working with open-source standards and containerization (Docker) to ensure data workloads can be ported across different cloud infrastructures.
· Collaborative Engineering: Proven track record of working within a team to successfully deliver and modernize large-scale data workloads.
· Ensure high levels of security, compliance, and data governance across data engineering processes.
Should you meet the requirements for this position, please email your CV to it.vacancy@stand-outstaffing.co.za. You can also contact the team on 031 350 4019 or alternatively you can visit our website www.stand-outstaffing.co.za
Should you not hear from us within 3 days, please consider your application unsuccessful.