Spark Engineer

Role:

The Data Engineer's primary role is to write Spark pipelines to ETL data within our data platform running on AWS. Working within a very dynamic team, this role must be able to help elevate the quality of the team. The Data Engineer will work closely with Data Analysts, Data Scientists, and business and technical teams to deliver outcomes that are secure, reliable, fault-tolerant, scalable, quality, and efficient.

Must Have Skills:
- Senior level Scala and Spark (MUST be actual development using SCALA - not pyspark or similar)
- Amazon EMR - Big Data processing
- Unit testing experience
- Must have experience with SQL and relational database systems (e.g., Oracle, SQL Server)

What will be my duties and responsibilities in this job?

1. Develop a scalable and resilient cloud data platform and scalable data pipelines.
2. Ensure industry best practices around data pipelines, metadata management, data quality, data governance, and data privacy.
3. Build highly scalable AWS Infrastructure (from scratch or through 3rd party products) to enable Big Data Processing in the platform.

Required Skills : Candidates MUST HAVE the following experience DOCUMENTED on resume: - Senior level Scala and Spark (MUST be actual development using SCALA - not pyspark or similar) - Amazon EMR - Big Data processing - Unit testing experience
Basic Qualification :
Additional Skills : - fully remote - highly visible project on one of the leading teams at Availity
Background Check :Yes
Drug Screen :Yes
Notes :
Selling points for candidate :- fully remote - highly visible project on one of the leading teams at Availity
Project Verification Info :
Candidate must be your W2 Employee :Yes
Exclusive to Apex :Yes
Face to face interview required :No
Candidate must be local :No
Candidate must be authorized to work without sponsorship ::No
Interview times set : :No
Type of project :Development/Engineering
Master Job Title :Dev: Other
Branch Code :Jacksonville

Location: Jacksonville, FL, US

Posted Date: 9/27/2024