Data Engineer
Optim-G Sourcing
- Centurion, Gauteng
- R15,000-18,000 per month
- Permanent
- Full-time
This includes data store design using different architecture patterns based on business requirements.Requirements:
- BCom Information Technology degree
- Data Engineering Tools (Azure Data Engineer ) certification
- ITIL or COBIT certification
- At least 2-4 years working exp in IT environment
- Exp with Phyton and SQL
- Exp with data pipelines, data lakes and Data Warehouses
- ETL processes
- Implement CI/CD pipelines
- Ensure the managed data (business intelligence) is fit for purpose relative to customer needs
- Design and develop end-to-end data solutions on the Data platform, ensuring optimal performance, scalability, and reliability.
- Collaborate with data architects, data analysts, and business stakeholders to understand data requirements, define data models, and implement data integration solutions.
- Implement data pipelines, data lakes, and data warehouses using.
- Develop and maintain ETL/ELT processes to extract, transform, and load data from various sources into Azure data platforms.
- Design and optimize data schemas, tables, and indexes for efficient data storage and retrieval.
- Perform data profiling, data cleansing, and data quality assessments to ensure data accuracy and consistency.
- Develop and implement data security and privacy measures, ensuring compliance with relevant regulations (e.g., POPIA).
- Monitor and troubleshoot data pipelines, identifying and resolving performance bottlenecks, data inconsistencies, and system issues.
- Collaborate with DevOps teams to implement CI/CD pipelines and ensure smooth deployment and release management of data solutions.
- Stay up to date with the latest data technologies, trends
Job Mail