Senior Data Engineer – Independent Contractor

Johannesburg, Gauteng
Contract
Full-time

23 days ago

Senior Data Engineer – Independent Contractor
Contract: 6 months (with possibility of renewal)
Location: Hybrid – Based in Bryanston, 2 days per week in the office
Start Date: ASAPOur client is seeking an independent contractor to join a strategic initiative to build an enterprise data platform aimed at transforming an entrepreneurship ecosystem through automated data capture and real-time analytics. This platform will consolidate data from five disparate systems to provide timely insights and drive impact.Key Responsibilities:

Build real-time ETL pipelines integrating Asana, SharePoint, AcceleratorApp, Fluxx, and Jibble.
Design and implement scalable data lake and warehouse architectures supporting over 300,000 beneficiaries.
Develop robust API connections with error handling and retry mechanisms.
Optimize data synchronization to occur within 5 minutes, processing 100,000+ records per hour.

Essential Skills:

5+ years experience with Python, Java, or Scala for data engineering
Proficiency with Apache Spark, Kafka, or similar streaming technologies
Experience with cloud platforms (Azure preferred, AWS acceptable)
Expertise in API integration and RESTful services
Skilled in SQL and NoSQL databases
Knowledge of data lake and warehouse architectures (e.g., Delta Lake, Databricks)
ETL/ELT pipeline development and real-time data processing

Desirable Skills:

Microsoft Azure Data Factory
Experience with CRM data models and impact measurement platforms
SharePoint API / Microsoft Graph API experience
Predictive analytics pipeline setup

Experience:

5 to 8 years in data engineering roles
Experience in non-profit or social impact sectors is advantageous
Proven success with multi-system integrations and large data volumes

Deliverables:

Fully functional, integrated data pipeline
Data lake architecture documentation
Automated data quality monitoring
Achievement of performance benchmarks (sync within 5 minutes)

If you’re an experienced independent contractor ready to make an impact through data, please apply with your CV and portfolio.Apply now!For more IT jobs, please visitIf you have not had any response in two weeks, please consider the vacancy application unsuccessful. Your profile will be kept on our database for any other suitable roles/positions.For more information contact:
Reinie Du Preez
Senior Specialist Recruitment Consultant
E-mail:For more information please contact:Reinie Du Preez

Network Recruitment

Apply Now