Full-Time
Senior Big Data Engineer
Deegit
No salary listed
Salt Lake City, UT, USA
In Person
Category
 Data & Analytics (1) |
|---|
Required Skills
Data Science
Apache Kafka
Java
Requirements
- BS in Computer Science, Information Systems, or related technical degree.
- 5 or more years’ experience with data transformation pipelines preferably Storm
- 5 or more years’ experience in messaging frameworks preferably Kafka.
- 5 or more years’ experience with Java programming.
Responsibilities
- Participate in the engineering and administration of big data systems.
- Apache Storm/Java development for both data transformation and augmentation.
- Employ best practices to ensure high availability and scalability.
- Install, configure, and maintain big data technologies and systems.
- Maintain documentation and troubleshooting playbooks.
- Respond to and resolve access and performance issues.
- Ability to work non-peak hours when needed.
- Assist in capacity planning.
- Proactively monitor performance.
- Collaborate with engineering teams to optimize data collection.
- Maintain change control and testing processes for all modifications and deployments.
- Conduct research and make recommendations on big data products, services, and standards.
Desired Qualifications
- Experience with operational analytics and log management using ELK or Splunk.
- Understanding and experience with SQL and MPP databases such as Greenplum.
- Experience with SaltStack or comparable deployment automation tools.
- Working knowledge of Zenoss, NewRelic, PagerDuty or other monitoring technologies.
- Experience with Apache Drill, Apache Hbase, Hive, Phoenix, or other Hadoop with SQL technologies.
- Experience working with batch-processing and tools in Hadoop technology stack (MapReduce, Yarm, Pig, Hive, HDFS)