Senior Data Engineer

Johannesburg, Gauteng
Permanent
Full-time

1 month ago

Job SummaryHire Resolve's client is seeking a highly skilled Senior Data Engineer to join their dynamic team within the telecommunications domain. The ideal candidate will be responsible for designing, developing, and maintaining scalable and reliable data pipelines to support the ingestion, processing, and analysis of Call Detail Records (CDRs) and real-time data streams.This role demands advanced technical expertise in stream processing frameworks such as Apache Kafka, Apache NiFi, Apache Flink, and Apache Druid, along with strong collaboration skills and a passion for data-driven solutions.Responsibilities:

Design and implement robust ETL/ELT pipelines for ingesting and transforming CDRs and telecom log data across multiple network types (2G/3G/4G/5G).
Develop and maintain real-time data pipelines using Apache Kafka, NiFi, and Flink.
Structure and integrate data for analytics platforms such as Apache Druid, Hudi, and Superset.
Manage large-scale distributed file systems and ensure efficient batch/stream data processing.
Build and manage CI/CD pipelines for data workflow deployments.
Optimize data performance, cost-efficiency, and reliability through continuous monitoring and enhancement.
Enforce data governance best practices including schema validation, access control, and regulatory compliance (e.g. GDPR).
Collaborate with development, DevOps, and analytics teams to deliver query-ready datasets for machine learning models and BI dashboards.
Implement role-based access control, data encryption, and secure identity management.
Design and maintain metadata repositories, data lineage, and asset discovery frameworks.
Build quality assurance and anomaly detection tools to monitor data integrity across all pipeline stages.
Support analytical modelling using OLAP systems and columnar databases.

Requirements:

Bachelor's Degree in Computer Science, Software Engineering, Computer Engineering, or related field.
Minimum of 5-8 years of experience in data engineering, preferably in telecom or large-scale distributed systems.
Strong expertise in Apache stack tools: Kafka, NiFi, Flink, Spark, Hive, Druid, Hudi.
Proficiency in Python, SQL, Shell scripting, and Linux-based systems.
Experience in database management: PostgreSQL, MySQL, and analytical querying.
Familiarity with data serialization formats (JSON, Avro, etc.) and secure data transfer protocols (SFTP).
In-depth knowledge of data quality control, lineage tracking, and data discovery tools.
Experience in full-text search and indexing of structured/unstructured data using Apache or similar tools.
Excellent problem-solving abilities, multitasking, and a fast-learning attitude.

How To Apply:

Contact Hire Resolve today for your next career-changing move
Our client is offering a highly competitive salary for this role based on experience.
Send your CV to:

or connect with Mischa Bornman via LinkedIn. * Alternatively, you can also contact me directly at Hire Resolve mischa.bornman@hireresolve.usWe will contact you telephonically in 3 days should you be suitable for this vacancy. If you are not suitable, we will put your CV on file and contact you regarding any future vacancies that arise.Hire ResolveIT/ComputerRecruiterAboutSpecialists in Civil, Structural, Mechanical Engineering, Information Technology, Mining, Manufacturing and Finance Careers! Hire Resolve is one of the larger and more agile South African recruitment companies that focus on placing professionals and skilled people in permanent employment and contract employment. We prefer and focus on working with top quality professionals and candidates in South Africa and Africa. Hire Resolve has successfully placed Engineering, Mining, IT, Manufacturing and Finance professionals with top firms across the Western Cape, Eastern Cape, KwaZulu Natal, Gauteng and in Africa. Hire Resolve has assisted candidates to find jobs at over 100 JSE listed companies of which many are global companies with offices and operations in South Africa and Africa. It is for this reason that we are well respected in the industries we operate in and in the recruitment industry.

Job Mail

Apply Now