Full-Time

Senior Software Engineer

Observability

Posted on 12/12/2024

Astronomer

Astronomer

201-500 employees

Data orchestration platform for pipeline management

Compensation Overview

$180k - $210kAnnually

+ Equity Component

Senior

Remote in USA

Category
Backend Engineering
Software Engineering
Required Skills
SQL
Go
Requirements
  • A strong focus on helping customers gain more visibility and actionable insights through our observability offerings.
  • Strong written and verbal communication skills, with the ability to find middle ground.
  • 5+ years of engineering experience.
  • Experience with fast-paced startup environments.
  • Experience with Golang.
  • Experience with SQL and OLAP databases.
  • Experience with distributed systems and stream processing.
  • Experience with software development best practices, such as code reviews, testing, CI/CD, version control, automation and debugging.
Responsibilities
  • Contribute to the design, development and vision of Astro’s data observability platform.
  • Collaborate with cross-functional teams to understand user requirements, implement and iterate on the observability and governance features to help our customers gain visibility into their data pipelines and data products.
  • Continuously evaluate and improve the architecture and implementation of our observability platform.
  • Contribute to the overall platform usability, reliability, and scalability.
Desired Qualifications
  • Experience with data observability products.
  • Experience with Apache Airflow or related workflow orchestrators

Astronomer.io provides a data orchestration platform that utilizes Apache Airflow to simplify the deployment of data pipelines. Its main product, Astro, helps businesses manage and monitor their data flows, allowing them to focus on delivering essential data pipelines. Astro centralizes visibility and control over data across various clouds and teams, supporting over 1500 integrations while ensuring enterprise-grade security. The platform is designed for data and machine learning teams in industries like finance and e-commerce, enabling them to maintain 100% uptime for their applications. Astronomer.io differentiates itself by offering tools that enhance compute efficiency, minimize scheduling delays, and accelerate the development and deployment of new pipelines, leading to a strong return on investment for clients. The goal of Astronomer.io is to empower organizations to optimize their data strategies and ensure the reliability of their data operations.

Company Size

201-500

Company Stage

Series C

Total Funding

$272.1M

Headquarters

Cincinnati, Ohio

Founded

2018

Simplify Jobs

Simplify's Take

What believers are saying

  • Astronomer's revenue grew 206% in H1 2023, driven by global demand for data orchestration.
  • Astro's AI-powered predictive analytics prevent pipeline failures, enhancing data reliability.
  • Astronomer's collaboration with Microsoft Azure offers seamless cloud-native data orchestration.

What critics are saying

  • Increased competition in the data operations market may dilute Astronomer's focus.
  • Dependency on Anyscale's distributed computing could pose risks if the partnership falters.
  • Rapid growth may strain Astronomer's operations, impacting service quality and customer satisfaction.

What makes Astronomer unique

  • Astronomer offers Astro, a modern data orchestration platform powered by Apache Airflow.
  • Astro provides real-time alerts and proactive monitoring for business-critical data pipelines.
  • Astronomer integrates orchestration and observability, reducing complexity in data infrastructure management.

Help us improve and share your feedback! Did you find this helpful?

Benefits

Remote Work Options

Company Equity

Growth & Insights and Company News

Headcount

6 month growth

0%

1 year growth

0%

2 year growth

-8%
VentureBeat
Feb 13th, 2025
Ai’S Biggest Obstacle? Data Reliability. Astronomer’S New Platform Tackles The Challenge

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More. Astronomer, the company behind Apache Airflow orchestration software, launched Astro Observe today, marking its expansion from a single-product company into the competitive data operations platform market. The move comes as enterprises struggle to operationalize their AI initiatives and maintain reliable data pipelines at scale.The new platform aims to help organizations monitor and troubleshoot their data workflows more effectively by combining orchestration and observability capabilities in a single solution. This consolidation could significantly reduce the complexity that many companies face when managing their data infrastructure.“Previously, our customers would have to come to us for orchestration data pipelines, and they’d have to go figure out a different data observability and Airflow observability vendor,” said Julian LaNeve, CTO of Astronomer, in an interview with VentureBeat. “We’re trying to make that a lot easier for our customers and give them everything in one platform.”AI-powered predictive analytics aims to prevent pipeline failuresA key differentiator of Astro Observe is its ability to predict potential pipeline failures before they impact business operations

Blockchain News
Oct 29th, 2024
Anyscale and Astronomer Collaborate to Enhance Scalable Machine Learning

This partnership allows organizations to effectively manage and scale their ML workflows by integrating Astronomer's workflow management capabilities with Anyscale's distributed computing power.

VentureBeat
Aug 15th, 2024
Do You Know Where Your Data Comes From? Apache Airflow Does And It’S Getting Updated To Advance Data Orchestration

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More. Getting data from where it is created to where it can be used effectively for data analytics and AI isn’t always a straight line. It’s the job of data orchestration technology like the open-source Apache Airflow project to help enable a data pipeline that gets data where it needs to be.Today the Apache Airflow project is set to release its 2.10 update, marking the project’s first major update since the Airflow 2.9 release back in April. Airflow 2.10 introduces hybrid execution, allowing organizations to optimize resource allocation across diverse workloads, from simple SQL queries to compute-intensive machine learning (ML) tasks. Enhanced lineage capabilities provide better visibility into data flows, crucial for governance and compliance

Artificial Intelligence News
May 8th, 2024
Accelerating ML application development: Production-ready Airflow integrations with critical AI tools

To support these new use-cases and to provide a starting point for Airflow users, Astronomer has worked with the Airflow Community to create Ask Astro-as a public reference implementation of RAG with Airflow for conversational AI.

VentureBeat
Dec 6th, 2023
Astronomer Gives Boost To Apache Airflow Platform Data Orchestration For Ai

Are you ready to bring more awareness to your brand? Consider becoming a sponsor for The AI Impact Tour. Learn more about the opportunities here. Getting data to and from different systems is often the domain of data orchestration. It is among the most widely used tools in the open-source Apache Airflow technology, originally created by Airbnb.Today Astronomer, the lead commercial sponsor behind the Apache Airflow project is out with its latest Astro platform update, providing enterprise support, security and management enhancements. While Airflow got its start helping with the orchestration of data pipelines for data analytics and business intelligence, the technology is now increasingly being used in support of artificial intelligence (AI) and machine learning (ML) workloads.“Airflow is very good at a couple of things, one is just basically writing and running data pipelines,” Julian LaNeve, CTO at Astronomer told VentureBeat. “Airflow lets you define pipelines as code, so you can do anything that the code will let you do which is essentially boundless.” Why Airflow is critical for modern data orchestrationLaNeve explained that Airflow has become popular in recent years as it enables organizations to define, build and deploy data pipelines more easily

INACTIVE