Data Engineer

• Design and develop scalable data pipelines and ETL processes to collect, process, and integrate data from various sources into a centralized data warehouse or data lake.
• Implement data models, schemas, and database structures to organize and optimize data storage for efficient data retrieval, writing, and analysis.
• Collaborate with quantitative and analysts to understand their data requirements and provide necessary infrastructure and tools for data exploration, analysis, and visualization.
• Ensure data quality and consistency by implementing data validation, cleaning, and transformation procedures.
• Monitor and maintain the performance, availability, and security of data systems, including database servers, data warehouses, and data processing frameworks.
• Troubleshoot and resolve data-related issues and performance bottlenecks in collaboration with the investment teams.
• Evaluate and integrate new data technologies, tools, and frameworks to improve data infrastructure and processing capabilities.
• Document data pipelines, workflows, and technical specifications to facilitate knowledge sharing and maintain systems.
• Stay up to date with industry trends, best practices, and emerging technologies in the field of data engineering.
• Assist and collaborate on team-wide projects.
• Contribute to the analytics platform from a Data Engineering perspective.
• Assist and investigate ad hoc data-related queries.

• Up to 3 years' experience with a specific focus on database management, data analysis, and producing ETL processes at scale is essential.
• Basic understanding of financial instruments and markets.
• Exceptional skill and experience in Python and SQL.
• Linux server administration.
• Git version control.
• Solid experience in managing and analysing large datasets.

• Matric / Grade 12 Essential
• University Post Graduate Degree (Computer Science/Information Systems/Mathematics/Stats/Business Science/Actuarial).

Careers24