We are looking for a Big Data Engineer (Hadoop & Cloud Specialist) to join our growing team. The ideal candidate will have deep expertise in the Hadoop ecosystem, Spark/PySpark, Python, and Cloud platforms (AWS, Cloudera, etc.). This role will focus on developing and deploying big data applications, providing advanced user support and troubleshooting, and driving platform optimization both on-premise and in the cloud. Beyond migration, the position will play a key role in onboarding, training, and long-term platform success, making it a vital contributor to our data strategy and business growth.Key ResponsibilitiesProvide advanced support and troubleshooting for users working on Hadoop, Spark, and PySpark environments.Develop, deploy, and maintain applications and processes on Hadoop clusters.Review, optimise, and enhance code and processes to improve performance, efficiency, and cost-effectiveness (especially in cloud environments).Support both on-premise and cloud-based big data environments, ensuring stability, scalability, and security.Contribute to the migration of big data workloads to the cloud, working closely with the ingestion team.Drive post-migration activities including onboarding of new teams, user training, and ongoing platform optimisation.Collaborate with cross-functional teams to identify gaps, provide technical guidance, and ensure best practices in big data development and operations.Required Skills & ExperienceStrong knowledge of the Hadoop ecosystem (HDFS, YARN, Hive, HBase, etc.).Proven experience with Apache Spark and PySpark for data processing.Proficiency in Python for data engineering and application development.Hands-on experience with Cloud platforms (AWS, Cloudera, or equivalent), including:Cloud NetworkingCloud SecurityCloud resource optimization for cost managementExperience reviewing and optimizing big data code and processes.Ability to design, build, and support scalable big data solutions in both on-prem and cloud environments.Strong troubleshooting and problem-solving skills.Preferred QualificationsExperience with data migration projects, particularly moving workloads from on-prem to cloud.Exposure to platform onboarding, user enablement, and training.Familiarity with performance tuning and cost optimization strategies in big data/cloud ecosystems.Developer-oriented background with the ability to contribute to long-term platform success and innovation.
