AI/ML Engineer - Python/PyTorch

Job Title - AI/ML Engineer Experience - 6 years Location - Hyderabad, Vizag Primary Skills - Python, PyTorch, TensorFlow, LLM Models Roles and Responsibilities : 1. Data Collection, Preprocessing, and Management : Data Collection : Gather diverse datasets for training LLMs or other machine learning models. This may include text data from various sources like web scraping, databases, or APIs. Data Preprocessing : Clean and preprocess raw text data (e.g., tokenization, stemming, lemmatization, removing stopwords) for NLP tasks using tools such as SpaCy, NLTK, or custom preprocessing pipelines. Data Augmentation : Create synthetic data or perform augmentation techniques to enrich training datasets, particularly in scenarios where large labeled datasets are scarce. Data Pipeline Development : Build automated and scalable data pipelines for continuous data ingestion, cleaning, and feeding into models. 2. Building and Training Machine Learning Models : Model Selection and Design : Design and implement deep learning architectures for specific use cases (e.g., LLMs for NLP tasks like sentiment analysis, text summarization, question answering). Model Development Using PyTorch and TensorFlow : PyTorch : Build and train custom neural networks using PyTorch, leveraging its dynamic computation graph and flexibility for research and experimentation. TensorFlow : Implement scalable, production-ready models using TensorFlow (including TensorFlow Hub and Keras for high-level model building). Training Large Models : Train large models like transformers (e.g., BERT, GPT, T5) using large-scale datasets. Efficiently handle high computational requirements for these models, potentially using cloud services (AWS, GCP) or GPUs. Fine-Tuning Pre-trained Models : Fine-tune pre-trained models like BERT, GPT-3, or other LLMs on task-specific data to improve performance on downstream applications. Model Evaluation : Use evaluation metrics like accuracy, F1 score, BLEU score (for text generation), or perplexity to assess model performance. Perform cross-validation and hyperparameter optimization. 3. Model Optimization and Scaling : Hyperparameter Tuning : Experiment with hyperparameters (e.g., learning rates, batch sizes, number of layers, dropout rates) to enhance model performance and prevent overfitting. Optimization : Use model optimization techniques such as quantization, pruning, and knowledge distillation to reduce the size and improve the inference speed of large models. Distributed Training : Implement distributed training using PyTorch Distributed or TensorFlow's MirroredStrategy to train large models efficiently across multiple GPUs/TPUs. 4. Model Deployment and Integration : Model Deployment : Deploy AI/ML models into production environments (e.g., AWS SageMaker, Google AI Platform) ensuring scalability, security, and robustness. API Development : Build APIs or microservices for serving models, enabling real-time predictions or batch processing using frameworks like Flask, FastAPI, or TensorFlow Serving. Model Monitoring : Implement monitoring systems to track the performance and accuracy of models in production. Detect model drift or degradation over time and retrain when necessary. Scalability and Optimization : Ensure that the models can scale to handle large-scale inference workloads. Use TensorFlow Lite for edge devices or ONNX for cross-framework deployment. 5. Research and Experimentation : Cutting-Edge Research : Stay up to date with the latest advancements in machine learning, especially in transformer models and NLP, and incorporate state-of-the-art techniques into your work. Innovation : Experiment with novel approaches for improving model accuracy, efficiency, or generalization (e.g., new transformer variants, unsupervised pretraining techniques). Contributing to Open Source : Contribute to or develop open-source projects that enhance machine learning tools, especially in the field of NLP and LLMs. (ref:hirist.tech)

Location: in, IN

Posted Date: 11/28/2024