AWS Data Engineer

Pretoria, Gauteng
Permanent
Full-time

2 months ago

Job SummaryOverviewAn exciting opportunity has become available for an experienced AWS Data Engineer. In this role, you'll be responsible for building and managing scalable data systems, from setting up data sources to integrating analytical tools, using AWS services.Key Responsibilities:

Data Architecture & Management: Design and maintain data systems using AWS services (e.g., AWS S3, AWS Glue, Athena). Organize data effectively and ensure easy access through data partitioning and cataloging strategies.
ETL Pipeline Development: Develop and optimize ETL (Extract, Transform, Load) pipelines using AWS Glue and PySpark. Focus on improving performance, scalability, and cost-efficiency in batch and real-time data processing.
Automation & Monitoring: Automate workflows and ensure they run efficiently. Set up monitoring and alerts for data pipelines, and optimize AWS resources for scalability and performance.
Security & Compliance: Follow security best practices, including API authentication, data encryption, and compliance with GDPR, HIPAA, and SOC2.
Collaboration & Mentorship: Work closely with data scientists, analysts, and backend teams to integrate data pipelines. Provide mentorship to junior team members and encourage growth and collaboration.
Quality Management: Ensure high-quality software development standards by adhering to best practices and complying with industry regulations like ISO, CE, and FDA.

Key Skills and Experience (5+ Years Required):

AWS Services: Experience with AWS Glue, S3, Lambda, Athena, and CloudWatch.
Data Engineering: Proven experience developing and optimizing ETL pipelines. Familiarity with SQL and NoSQL databases (e.g., PostgreSQL, MySQL, DynamoDB).
Programming: Strong skills in Python, particularly using PySpark with AWS Glue.
Data Modeling & Optimization: Experience in data modeling, schema design, and database optimization.
Machine Learning: Experience integrating data pipelines with machine learning workflows and deploying models with AWS SageMaker.
Compliance & Security: Knowledge of data governance, API security, and compliance with industry standards and regulations.

Education & Certification Requirements:

Bachelor's degree in Computer Science, Engineering, or a related field.
AWS certifications, such as AWS Certified Data Engineer, Solutions Architect, or Data Analyst, are highly desirable.

CodeConnect Staffing (Pty) LtdRecruiter

Job Mail

Apply Now