Data Engineer - Google Cloud Platform

Job Description : We are seeking a skilled and motivated GCP Data Engineer to join our dynamic team. The ideal candidate will have strong experience with Google Cloud Platform (GCP) and a deep understanding of data engineering principles. You will be responsible for designing, developing, and maintaining scalable data pipelines and systems using GCP tools. Key Responsibilities : - Design, develop, and maintain data pipelines and ETL processes on GCP. - Work with large datasets, ensuring high performance and scalability. - Utilize BigQuery for data storage, querying, and analysis. - Implement data workflows using Airflow or Dataflow. - Develop and maintain scripts in Python for data manipulation and automation. - Optimize SQL queries and database performance for efficient data retrieval. - Collaborate with data scientists, analysts, and other engineers to meet data needs. - Ensure data quality, consistency, and integrity across systems. - Monitor and troubleshoot data systems, ensuring high availability and reliability. Mandatory Skills and Qualifications : - Google Cloud Platform (GCP) : Hands-on experience with GCP services, including BigQuery, Cloud Storage, and Pub/Sub. - Python : Proficiency in Python for scripting, data manipulation, and automation. - SQL : Strong SQL skills, with experience in writing and optimizing complex queries. - BigQuery : Deep understanding of BigQuery architecture, including query optimization and partitioning. - Airflow or Dataflow : Experience with setting up and managing workflows using Apache Airflow or Google Cloud Dataflow. - Data Engineering : Solid understanding of data warehousing concepts, ETL processes, and data modeling. - Version Control : Familiarity with version control systems like Git. Preferred Qualifications : - Experience with other GCP services such as Data Studio, Cloud Functions, or Cloud Composer. - Knowledge of CI/CD pipelines and infrastructure as code (IaC) using tools like Terraform. - Familiarity with data governance and security best practices. - Certification in GCP Data Engineering. Job Types : Full-time, Permanent Experience : Total work : 4 years (Preferred) GCP : 2 years (Preferred) Bigquery : 1 year (Preferred) (ref:hirist.tech)

Location: chennai, IN

Posted Date: 11/21/2024