A

Adthena

AI-driven search intelligence and competitive analysis

Senior Anti-Bot Scraping Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeRemote in UK

About the Job

Requirements

  • Strong professional experience with Python or another main programming language, such as Go, Rust, or JavaScript, including production-grade scraping systems.
  • Deep hands-on experience scraping Google or similarly protected platforms.
  • Solid understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behaviour.
  • Experience with browser automation frameworks such as Playwright, Selenium, Puppeteer, or equivalent.
  • Strong knowledge of HTML parsing, DOM traversal, and data extraction techniques.
  • Experience handling rate limiting, CAPTCHA, IP rotation, and bot detection systems.
  • Comfort working with asynchronous and concurrent scraping architectures.
  • Experience running scrapers at scale in cloud environments.
  • Strong debugging skills and ability to reason about complex failure modes.
  • Experience with headless browser fingerprinting and anti-bot evasion techniques.

Responsibilities

  • Design and maintain large-scale Python scraping systems that operate reliably against Google and other highly defended websites.
  • Build scrapers for dynamic, JavaScript-heavy pages using browser automation and hybrid approaches.
  • Continuously adapt to changes in page structure, request flows, and anti-automation mechanisms.
  • Engineer robust extraction pipelines that prioritize data accuracy, consistency, and observability.
  • Implement proxy, fingerprinting, and request-routing strategies to maximize stability and throughput.
  • Monitor scrape health, detect failures early, and rapidly debug production issues.
  • Optimize performance, cost, and latency across scraping infrastructure.
  • Work closely with data engineers, data scientists, and product teams to ensure scraped data is usable and trusted.
  • Document scraping logic and operational runbooks for long-term maintainability.

Desired Qualifications

  • Familiarity with containerization and orchestration, such as Docker and Kubernetes.
  • Experience with distributed task systems and job queues.
  • Background in data quality monitoring and anomaly detection.
  • Experience working with search, advertising, or competitive intelligence data.

About the Company

Adthena

AI-driven search intelligence and competitive analysis

Adthena provides AI-powered search intelligence and competitive analysis for digital marketing. It offers a suite of tools that help marketing teams optimize campaigns, defend brand equity, and plan market entry by analyzing comprehensive market data and monitoring changes in search performance. The product works by using AI to categorize and translate vast data into actionable insights, enabling users to measure market share, compare against industry benchmarks, and receive guidance for campaign optimization and strategic growth. What sets Adthena apart is its whole-market view—covering insights across industries and markets—combined with subscription-based access and tailored consultancy, giving clients both automated analytics and expert guidance. The company's goal is to boost clients’ ROI and performance while enabling confident expansion into new markets and better protection of their brand.

Company size
51-200
Company stage
Debt Financing
Total funding
$36.6M
Headquarters
London, United Kingdom
Founded
2012

Recently Posted Jobs

Explore more opportunities at Adthena

See more jobs