Principal Software Engineer - Guest Experience, Site Reliability

** This role is not eligible for sponsorship AND is four days onsite hybrid at our N. Scottsdale office **

Who are we looking for?

Choice Hotels, one of the world’s largest lodging franchisors, has an exciting new opportunity as a Principal Software Engineer on our Digital Site Reliability team in Digital Experience. The Digital Site Reliability team keeps the guest-facing platforms that millions of travelers rely on available, fast, and trustworthy: choicehotels.com, the Choice Hotels Android and iOS apps, and the call center technologies our reservation agents use every day. These platforms are highly visible, revenue-generating, and central to how guests book stays, manage reservations, and engage with our loyalty program.

As a key member of this team, you will be the engineer people turn to when something complex breaks and the clock is ticking. You will dig into logs and telemetry, build the dashboards that make problems visible before guests feel them, and design lasting solutions to highly complex, multi-variable problems that span front end, API, cloud, and mobile layers. You will write production code, but your real signature is problem solving: finding the root cause, framing the options, and bringing the right people together to decide quickly. You will also use AI-assisted tools pragmatically to speed up diagnostics, reduce toil, and improve outcomes.

If you are a self-driven engineer who can code with the best of them, takes pride in untangling the hardest problems, and loves leading a room through a high-pressure incident, we invite you to apply today and #MakeItYourChoice.

Your Responsibilities

• Troubleshoot and resolve complex production issues across choicehotels.com, Choice’s mobile apps, and call center technologies by analyzing logs, metrics, traces, and runtime signals, with Grafana as a primary tool.

• Lead and facilitate cross-functional conversations during incidents and time-sensitive technical discussions, aligning engineering, product, and business partners on root cause, options, and next steps.

• Participate in the on-call rotation for Guest platforms, serving as a senior escalation point and driving critical incidents to resolution.

• Write secure, stable, maintainable code to fix reliability defects, build tooling and automation, and strengthen the resiliency of our applications.

• Apply expertise in CDN, DNS, WAF, security, and vulnerability management to protect the environment, with experience using solutions such as Akamai.

• Drive improvements to CI/CD pipelines and release safety in Harness, including automated deployment verification, rollback strategies, and pipeline guardrails.

• Identify systemic risks and failure modes in AWS-hosted services, such as capacity limits, brittle error handling, poor failure propagation, and unsafe defaults, and translate them into well-scoped improvements.

• Apply AI-assisted development, log analysis, and incident tools where they measurably improve root cause analysis, engineering productivity, and reliability outcomes.

• Conduct trade-off analysis across alternative technical approaches and present clear pros, cons, and recommendations to peers and leadership.

• Mentor and coach engineers through code reviews, pairing, and knowledge sharing, and influence engineering practices across teams without formal authority.

• Partner with Solution Architects, Engineering Managers, and Mobile, Web, and Contact Center teams to define non-functional requirements for scalability, availability, and security.

• Evaluate and introduce tools and technologies that improve observability, platform safety, and developer productivity, prioritizing simplicity, adoption, and long-term impact.

Your Experience, Skills & Competencies

• Bachelor’s degree in Computer Science or a related technical field, or equivalent practical experience building and operating production systems. Master’s degree or equivalent experience preferred.

• 10+ years of relevant experience in software development, including experience with distributed systems, cloud platforms, and API-based architecture, and experience serving as a technical lead or in an architecture role.

• Strong hands-on experience with AWS public cloud services and an understanding of how managed services influence reliability, scalability, and operational behavior.

• Hands-on experience building and maintaining CI/CD pipelines in Harness (Jenkins, GitHub Actions, or similar a plus), with Git-based source control and sound branching and collaboration practices.

• Strong Java experience, including building and operating Java-based services (Spring Boot preferred) in cloud and containerized environments.

• Proficiency building front-end applications with React and modern JavaScript or TypeScript.

• Experience designing, consuming, and troubleshooting GraphQL APIs, plus familiarity with RESTful APIs and OpenAPI specifications.

• Proven experience diagnosing complex production issues using observability platforms and centralized logging. Hands-on Grafana experience, including log exploration and dashboard creation, is required; experience with tools such as AppDynamics, OpenSearch, Amazon CloudWatch, or similar is a plus.

• Demonstrated experience using AI-assisted tools (such as code assistants, log and trace analysis, or incident analysis tools) to improve engineering effectiveness or system reliability.

• Solid understanding of Site Reliability Engineering principles, including SLOs, error budgets, incident management, and blameless postmortems.

• Experience participating in an on-call rotation and leading critical incident response for high-traffic, consumer-facing systems.

• Working knowledge of relational and non-relational data stores and how persistence, availability, and failure characteristics affect system resiliency.

• Strong knowledge of application security best practices and awareness of relevant compliance requirements (such as PCI, OAuth, and OWASP).

• Excellent written and verbal communication, with the ability to explain complex technical issues to audiences ranging from engineers to senior leaders and to facilitate productive discussion under time pressure.

• Ability to deal with ambiguity, take ownership of highly complex problems from inception to completion, and lead others in the absence of formal positional power.

• Preferred: Kotlin experience and experience supporting mobile applications (Android and/or iOS), including mobile release, crash, and performance monitoring.

• Preferred: AWS certifications or equivalent technology certifications.
• Competencies demonstrated: Technical Leadership, Problem Solving & Decision Quality, Manages Complexity, Collaboration & Communication.

Your Team

This is an individual contributor role that will report to the Head of Digital & Loyalty Engineering. You will collaborate closely with Software Engineers, Solution Architects, Engineering Managers, and the web, mobile, and call center teams across Digital Experience to keep Choice’s Guest platforms reliable and performing at their best.

** This role is not eligible for sponsorship **

Choice prioritizes our associate wellbeing by offering a comprehensive benefits program that is both competitive and flexible to help you achieve your wellbeing goals - here are just a few:

• Competitive compensation and benefits, including medical, dental, and vision coverage
• Leave and paid time-off for holidays, vacation, personal, family, volunteer, sick, jury duty, bereavement, military, and religious observance
• Financial benefits for retirement and health savings
• Employee recognition programs
• Discounts at Choice hotels worldwide

About Choice

Choice Hotels International, Inc. (NYSE: CHH), is one of the largest lodging franchisors in the world. With 7,500 hotels in 45+ countries and territories, we offer a range of high-quality lodging options in the upper upscale, upper midscale, midscale, extended stay, and economy segments. We’re the hotel company for those who choose to bet on themselves — the striver, the dreamer, the entrepreneur — because that’s who we are, too.

At Choice, we are united by the simple belief that tomorrow will be even better than today — for associates, our company, and our franchisees. At our worldwide corporate headquarters in North Bethesda, Maryland, at our technology center in Scottsdale, Arizona, and through our associates around the globe, every voice is heard and every idea is listened to, no matter what area of the company they come from. We are united in supporting the entrepreneurial dreams of our thousands of franchise owners, which propels us forward — giving our work at Choice a purpose larger than our business.

Our corporate office locations:

North Bethesda, MD — Located at Pike & Rose, our worldwide headquarters is less than 15 miles from Washington, D.C., one block away from the North Bethesda Metro station, with easy access to I-495, complimentary parking, electronic charging stations, restaurants and retail.

Scottsdale, AZ — Located at the northwest corner of Loop 101, the Scottsdale office is home to our technology, eCommerce and customer service organizations, with easy access to complimentary parking, electronic charging stations, restaurants and retail.

Minneapolis, MN — Select roles are based in our Minneapolis office on Highway 394, near the intersection with Highway 100, only five minutes from downtown.

Field/Remote — Select roles designated as field/remote will require associates to work from a home office, connecting virtually with Choice team members and leadership on Zoom, with possible required travel depending on the role.  

Choice’s Cultural Values

Welcome and Respect Everyone | Be Bold | Be Quick | Listen | Be Curious | Show Integrity

Choice’s Leadership Principles

Act with Intention | Lead with Authenticity | Grow & Deliver