Ospyn - Data Engineer - Apache NiFi/Talend

Description : - Design, develop, and maintain scalable data pipelines to support data-driven initiatives. - Integrate data using ETL/ELT processes and tools such as Apache NiFi, Talend, Informatica, Apache Airflow, or DBT. - Implement and optimize data models and techniques to support data integrity and performance. - Utilize big data technologies to process and analyze large volumes of data efficiently. - Ensure data governance and implement measures for data quality and security. - Manage on-premises infrastructure including server management, networking, storage, and OS platforms (Linux, Windows Server). - Implement cloud security best practices to secure data and infrastructure. - Utilize cloud compute and storage services to deploy and manage scalable data solutions. - Collaborate effectively with cross-functional teams to understand data requirements and deliver scalable data solutions. Preferred Skills : - Proficiency in SQL and NoSQL databases. - Experience with ETL/ELT processes and tools (e.g., Apache NiFi, Talend, Informatica, Apache Airflow, DBT). - Strong programming skills in Python and SQL; familiarity with Java, Scala, or R is a plus. - Knowledge of data modeling techniques (e.g., star schema, snowflake schema, normalization, denormalization). - Experience with big data technologies (e.g., Hadoop ecosystem, Spark, Kafka). - Implementing data governance and ensuring data quality. - On-premises infrastructure management (e.g., server management, networking, storage, Linux, Windows Server). - Experience with on-premises cluster management (e.g., Cloudera, Hortonworks). - Implementing on-premises security measures is a plus (e.g., firewalls, VPNs, data encryption). - Proficiency with Azure data services (e.g., Azure Data Factory, Azure SQL Database, Azure Synapse Analytics, Azure Databricks, Azure Data Lake Storage). - Experience with AWS data services (e.g., AWS Glue, Amazon Redshift, Amazon RDS, Amazon S3, AWS Lambda, Amazon EMR). - Proficiency with GCP data services (e.g., BigQuery, Cloud Dataflow, Cloud Dataproc, Cloud Pub/Sub, Cloud Storage). - Implementing cloud security best practices (Azure AD, AWS IAM, GCP IAM, Key Vaults, VPCs). - Experience with cloud compute and storage services (Azure VMs, EC2, GCP Compute Engine, Azure Blob Storage, EBS, Cloud Storage). - Strong problem-solving, communication, and collaboration skills. (ref:hirist.tech)

Location: trivandrum, IN

Posted Date: 11/23/2024