S

Solvd

AI-driven software development and QA services

Senior Data Engineer - Knowledge Ingestion

Data EngineeringPosted on Nov 13
Full-timeRemote in USANo salary listedRemote

About the Job

Requirements

  • At least 4 years of experience building production data pipelines, including at least 2 years using AWS with Python.
  • Hands-on experience with Microsoft Graph API for SharePoint and Teams content, including delta queries, permissions, and throttling, or equivalent enterprise content platform experience.
  • Experience landing unstructured content such as HTML, Office documents, PDFs, and chat transcripts into S3 and preparing it for search or vector indexing.
  • Working knowledge of Amazon Bedrock Knowledge Bases data sources and sync behavior, or a comparable vector ingestion path on OpenSearch.
  • Infrastructure-as-code experience with Terraform or AWS CDK, and CI experience for deploying pipelines into a client-owned account.
  • Ability to work from a shared backlog in client repositories and document work as it is performed.

Responsibilities

  • Build the iKNOW connector against Oracle B2C Service knowledge APIs, including a full inventory extract and incremental sync by modified date, capturing article status and answer version.
  • Build SharePoint, OneDrive, and Teams connectors using Microsoft Graph, including site and drive enumeration, delta queries for incremental changes, and capturing sharing and group membership as permission metadata on every item.
  • Build the Sitecore connector against its content APIs for in-scope public web content.
  • Implement the ingestion pipeline, landing content in S3 with per-source prefixes, converting documents and extracting text, deduplicating by content hash, enriching metadata from taxonomy, and syncing Bedrock Knowledge Base data sources with architect-set chunking parameters.
  • Produce inventory extracts for content disposition reviews and counts for end-of-Phase-1 re-baselining.
  • Implement retirement handling so items marked retired or deleted at source are removed from the index on the next sync.
  • Handle throttling and backoff for Oracle and Microsoft APIs, idempotent reruns, and per-source failure isolation so an outage in one connector does not stall the others.
  • Instrument every pipeline stage with CloudWatch metrics and write ingestion runbooks.
  • Pair with client IT during weeks 7–10 so they can run, rerun, and extend the pipelines after transition.

Desired Qualifications

  • AWS Certified Data Engineer Associate certification.
  • Prior integration experience with Oracle B2C Service or another Oracle Service Cloud product.
  • Prior integration experience with Sitecore XM or XP content APIs.
  • Experience with AWS Glue, Step Functions, or EventBridge for orchestration, and document-parsing libraries for Office and PDF formats.

About the Company

Solvd

AI-driven software development and QA services

Solvd is a global B2B technology partner that provides strategic digital transformation and software engineering services, including AI advisory, data engineering, digital experience design, custom application development, DevOps and cloud engineering, and quality engineering. Its Zebrunner platform is an AI-powered quality management suite that unifies manual and automated testing with test case management, AI-driven failure analysis, dashboards, and integrations with Jira and Slack, across real devices and multiple browsers. The company differentiates itself by blending AI strategy with practical engineering and ongoing quality assurance, supported by a global delivery model and acquisitions of EastBanc Technologies and Tooploox to deepen AI and enterprise transformation capabilities. Its goal is to help Fortune 500 and other large organizations digitally transform efficiently, improve software quality, and speed up product delivery.

Company size
501-1,000
Company stage
Not listed
Total funding
Not listed
Headquarters
Roseville, CA
Founded
2011

Recently Posted Jobs

Explore more opportunities at Solvd

See more jobs