Kloud9 - Data Engineering Manager - Cloud Platform

Job Title : Data Engineering Manager (Remote) Location : Remote Position Type : Full-time / Consultant Experience : 10-12 yrs Job Description : We are seeking an experienced Data Engineering Manager who is hands-on in managing and delivering complex cloud-based data engineering projects, including data lake implementations. The ideal candidate will lead engagements from pre-sales through to delivery, ensuring high-quality outcomes in terms of data architecture, storage, transformation, and analytics. This role requires a strong technical background in modern cloud technologies, data governance, and quality : - Lead the design and implementation of large-scale data engineering projects, including data lakes and data pipelines on cloud platforms like AWS, Azure, or GCP. - Drive the pre-sales process by engaging with clients, understanding requirements, and developing technical proposals and proof of concepts (PoCs). - Architect scalable and secure data storage solutions using technologies like Amazon S3, Azure Data Lake Storage, and Google Cloud Storage. - Oversee the development of ETL/ELT pipelines using tools such as AWS Glue, Apache Spark, Databricks, or Azure Data Factory. - Ensure efficient data transformation and quality assurance processes by leveraging tools like AWS Lambda, Google Cloud Functions, or Azure Functions for serverless computing. - Implement data governance frameworks to ensure data quality and compliance using services like AWS Lake Formation, Azure Purview, or Google Data Catalog. - Collaborate with data scientists, BI developers, and analytics teams to ensure the smooth flow of data and insights across the organization. - Manage a team of data engineers, providing technical guidance and mentorship throughout the project lifecycle. - Engage with stakeholders and clients to align project deliverables with business goals. - Monitor and optimize data processing and storage to ensure efficiency and Skills : - 5 years of hands-on experience in data engineering, including leading data lake implementations and cloud-based solutions. - Proficiency in cloud platforms (AWS, Azure, or GCP) and services like Amazon S3, Azure Data Lake, Google Cloud Storage. - Extensive experience with data transformation tools such as Apache Spark, Databricks, AWS Glue, and Azure Data Factory. - Expertise in serverless architectures (e.g., AWS Lambda, Google Cloud Functions, Azure Functions). - Strong understanding of data governance, data quality, and security best practices in the cloud. - Familiarity with containerization and orchestration technologies such as Kubernetes and Docker. - Proficiency in SQL, Python, Scala, or Java for data manipulation and processing. - Experience working with data warehousing and analytics solutions like Amazon Redshift, Google BigQuery, or Azure Synapse. - Strong leadership and project management skills, with a track record of managing teams and delivering complex projects from pre-sales to delivery. - Excellent communication skills to effectively interact with stakeholders, clients, and technical teams. Preferred Qualifications : - Experience with data governance tools like AWS Lake Formation, Azure Purview, or Google Data Catalog. - Certifications in cloud platforms such as AWS Certified Solutions Architect, Azure Data Engineer, or Google Professional Data Engineer. - Familiarity with CI/CD pipelines and DevOps practices in the context of data engineering. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/28/2024