Facebook pixel

Data Engineer II
Open Web Collections, Python & Web Scraping
Posted on 9/2/2022
INACTIVE
Locations
Remote • United States
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Data Analysis
Data Structures & Algorithms
MongoDB
Python
Writing
Requirements
  • 2+ years of python programming: You are comfortable writing production-grade data processing infrastructure in python. Your debugging skills are strong
  • Data skills: You are comfortable working with large, complex data structures. You have experience transforming large datasets in python, querying with elastic search (MongoDB) and developing corresponding APIs
  • Well versed in both computer programing and data engineering: You want to review as well as question existing code, and can suggest modifications
Responsibilities
  • You will lead development, productionalization and maintenance of data extraction via custom page harvesters, APIs and RSS feeds
  • Create well monitored production data processing pipelines which convert raw data into architected structured data
  • Maintain and upgrade existing data flow production systems
  • Collaborate with junior engineers and senior engineers through design reviews, code reviews, pair programming, and other software engineering best practice collaboration
Recorded Future
Intelligence company