Kafka Administrator - On Premises Environment

Responsibilities : 1.Kafka Cluster Management : a. Install, configure, and maintain Confluent Kafka clusters in an on-premises environment. b. Manage Kafka brokers, Zookeeper, Connect, Kafka Streams, and related services. c. Monitor and tune the performance of Kafka clusters to meet operational SLAs. 2.Security & Compliance : a. Implement and manage Kafka security including SSL/TLS, ACLs, role-based access control (RBAC). b. Ensure that Kafka infrastructure complies with the organization's security policies and best practices. 3. Monitoring & Alerting : a. Set up monitoring and alerting for Kafka clusters using tools like Prometheus, Grafana, or Confluent Control Center. b. Analyze logs and metrics to proactively detect and resolve issues. 4. Automation : a. Develop automation scripts and frameworks to streamline Kafka operations, such as provisioning, configuration, scaling, and monitoring using tools like Ansible, Terraform, or custom Python scripts. b. Automate Kafka cluster deployments, upgrades, and patch management. c. Implement auto-scaling, failover, and disaster recovery solutions through automation. d. Knowledge of CI/CD pipelines and infrastructure-as-code (IaC) concepts for Kafka deployments. 5. Maintenance & Upgrades : a. Regularly perform maintenance and upgrades for Kafka components (brokers, Zookeeper, etc.). b. Handle Kafka backup and disaster recovery procedures. 6. Troubleshooting & Support : a. Provide operational support and incident management for Kafka-related issues. b. Work closely with developers and data engineers to resolve application-level issues. 7. Documentation : a. Maintain accurate documentation of Kafka architecture, configuration, and operational procedures. b. Create runbooks for various Kafka administrative tasks. 8. Experience : a. 3-5 years of hands-on experience managing Kafka clusters in production environments using Kubernetes. b. Solid understanding of distributed systems, high availability, and failover mechanisms. c. Experience with Kafka Cluster Linking and cross-cluster replication. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 11/28/2024