Data Engineer - Generative AI

Key Responsibilities : - Design, develop, and maintain scalable data pipelines to support Generative AI applications. - Work with large datasets and ensure efficient data extraction, transformation, and loading (ETL) to enable AI model training and inference. - Collaborate with data scientists and AI engineers to prepare high-quality data for Generative AI models (e.g., GANs, transformers). - Develop and optimize Python scripts and tools to automate data processing and support machine learning workflows. - Utilize Azure services (e.g., Azure Data Lake, Azure SQL Database, Azure Databricks) to build cloud-based data infrastructure. - Implement best practices for data governance, data quality, and data security in line with company policies. - Perform data wrangling and cleaning tasks to ensure high-quality datasets for AI applications. - Monitor and troubleshoot data pipelines and workflows to ensure they run efficiently and reliably. - Leverage Azure Machine Learning, Azure Synapse Analytics, and other Azure tools for data engineering tasks and integration with AI workflows. - Stay updated on the latest advancements in Generative AI and data engineering practices. - Document processes, workflows, and data models to ensure scalability and maintainability. Preferred Skills : - Certifications in Azure (e.g., Azure Data Engineer, Azure AI Engineer) or relevant data engineering platforms. - Experience in deploying Generative AI models to production environments. - Familiarity with big data technologies (e.g., Apache Kafka, Apache Flink). - Knowledge of DevOps practices for machine learning and AI model deployment. - Experience with version control tools like Git. (ref:hirist.tech)

Location: in, IN

Posted Date: 12/27/2024