Python Infrastructure Developer

A great opportunity for an experienced Python Infrastructure Developer, working with a leading international invest bank.

The Role

A fantastic opportunity for somebody with 3+ years python experience to work with state of the art technologies to deliver industry leading solutions in the Telemetry, Observability and Monitoring space. You would work with a team of enthusiastic forward thinking SREs & Engineers who are working to radically transform how the Bank manages the operation of its estate. They’re striving for visibility of all their key services in a single observability platform and then leverage that telemetry to best manage their services from infrastructure through to the business applications. This is the start of our evolution towards a full AI Ops Capability with automated anomaly detection, automated impact and root cause analysis and generated resolutions.

You will be a part of their Observability on-boarding task force working with global Infrastructure teams to fast track their adoption of best Observability practices and tools. This includes the collection of metrics of their key infrastructure services and the introduction of SLI/SLOs to track service levels and availability. These goals will be achieved by leveraging the Observability toolsets and writing python to build custom integrations to existing data sources. For example, writing python to pull key metrics from the corporate network monitoring tools via API and feeding those in to the central Observability platform, then leveraging that telemetry to define SLI/SLOs. They want to work on enhancing their configuration management database so they can traverse the schema to instantly determine potential impact of any service level degradation.

The observability platform consists of tools from vendor, open source and in house. These tools include Grafana UI, Loki (logs), Mimir (like Prometheus), Tempo (traces), Sloth, RightITNow, EverBridge and Open Telemetry.  Experience of the Observability principles and the Grafana toolset would be a big advantage. Their solutions are deployed on Linux backend, thus intermediate knowledge of Linux is a must. You must be able to demonstrate successful delivery of projects from requirement gathering through to delivery of the production deployed solution.

As the Python Infrastructure Developer, you must have:

• Python (3 + years active development) – Intermediate to expert level python development including experience of multi-threading for scalability
• At least 2 years’ hands-on on Linux systems from a hands on user perspective. Strong hold on general unix commands and scripting.

Ideally you will have;

• Experience of working in the Telemetry and Observability Space using industry recognised tools.
• Advantage to have experience of the Grafana toolsets used at the bank, which include Grafana Loki, Mimir and Tempo and the Grafana UI.
• Understanding of Open Telemetry and Prometheus Client Side Integrations
• Continuous Integration / Deployment via DevOps Tools such as GitLab, Jenkins, Ansible, Nexus etc.
• Collaboration Tools – Confluence / JIRA / Microsoft Teams & Office 365
• Basic awareness / familiarity with other infrastructure technologies such as Middleware (ActiveMQ / Solace / EMS / Tibco etc.), Database Servers, Web servers, Load balancers, Directory Services etc. This will help when working with the domain experts in each infrastructure areas.
• Solid understanding of release, deployment, and change management processes
• Delivering to a medium / large scale production environment