Senior Data Analyst
College Board – Learning & Assessments (L&A) Division
Location: This is a fully remote role. Candidates who live near CB offices have the option of being fully remote or hybrid (Tuesday and Wednesday in office).
Type: This is a full-time position
About the Team
The Data Management & Operational Support (DMOS) team serves as the gatekeeper and steward for all data—including content, metadata, and statistics—coming in and out of Assessment Design & Development (AD&D) to downstream stakeholders. We guide the department in adopting best data practices and partner with Test Security to maintain the integrity and security of all our test content—both within the organization and in the testing field. We ensure, where possible, the highest degree of quality of our data so the student testing experience is flawless and their score accuracy is upheld. We partner with Psychometrics, Applied Psychometrics Research, and Math and Reading & Writing subject matter experts to conduct intensive data analyses that inform evaluation of the efficacy and validity of our assessment instruments, and support the efforts of the State Work team to increase adoption of the digital SAT Suite at state and district levels. In addition, we are both key stakeholders and consultants on development of College Board’s end-to-end digital assessment ecosystem, from our item development platforms to digital delivery, scoring, item calibration, and reporting.
About the Opportunity
As our Senior Data Analyst, you are recognized as a subject matter expert for all data—including content, metadata, and statistics—coming in and out of Assessment Design & Development. You work closely with colleagues in Content, Applied Psychometrics Research, Psychometrics, and Research to understand their business needs and are responsible for analyzing and interpreting large amounts of student SAT test data to guide data-driven strategy at College Board, inform ongoing assessment item development and pretesting, and improve the student test experience. You work on complex problems where analysis of situations or data requires an in-depth evaluation of multiple factors and frequently contributes to the development of new business strategy, ideas, and methods. You exercise significant independent judgment to determine the best method for accomplishing work and achieving objectives in a fast-paced and deadline-driven environment.
In this role, you will be responsible for:
Assessment Research Support (70%)
- Conduct comprehensive data analysis and develop tools to support innovative AI projects, including automated essay scoring (AES) and pretest item statistics prediction. These initiatives aim to significantly increase scoring efficiency and streamline the item development process, resulting in substantial cost reductions for human reading in SAT essays and AP free response questions and lightened sample size requirements for pretesting.
- Fulfill both ad hoc and planned data requests — from various stakeholders internally at AD&D, Psychometrics, Research, Data Science and/or downstream systems — related to SAT Item metadata, statistics and usage, and student test question timing data from all SAT administrations.
- Develop and implement strategies for making data available to AD&D Content staff, Applied Psychometric Research, Psychometrics, and others, following established processes and protocols to ensure data accuracy and consistency in reporting
- Collect, load, clean, profile, and analyze large amounts of student testing data
- Present findings and analysis to requestors and stakeholders across the organization, and lead any follow-up analysis or reporting requests
Assessment Operational Support (30%)
- Conduct analysis and provide descriptive data-driven reports to stakeholders in support of SAT test assembly and pretest design; e.g., aggregate summaries, frequencies, rosters, cross-tabulations, correlations, and regression analyses
- Participate in item challenge investigations alongside DMOS team members, from triaging initial challenge submission through disposition and communication back to the challenge issuer (student)
About you, you have:
- Ideally 10+ years of experience in supporting assessment content development workflow
- Expertise in data exploration using SQL including ability to create complex queries
- Proficiency in programming languages such as Python or R to integrate with machine learning (ML) algorithms and large language models (LLMs) and predictive modeling techniques to enhance research and development
- Experience with relational data modeling and metadata management, leveraging relational databases and/or data warehouse to drive business reporting solutions
- Excellent attention to detail and proven ability to break down and convey complex information to both technical and non-technical audiences effectively.
- Must be resourceful, able to adapt to new situations, and to work in a fast-paced and deadline-driven environment
- Ability to take initiative without established guidelines and procedures using significant independent judgment.
- Advanced critical thinking and problem-solving skills and the ability to work independently as well as work well in group problem solving situations.
- Ability to leverage various modes of documentation and presentation — including diagrams, charts, and in written form — to communicate with stakeholders at multiple levels.
- Experience with gathering and analyzing technical and data requirements
- Excellent MS Excel, Word, Smartsheet skills
- Familiarity with data cataloguing tools and best practices
- Experience using standards-based web technologies such as HTML, XML, CSS
- A passion for supporting educational and career opportunities for millions of students
- The ability to travel 3-5 times a year to College Board offices or on behalf of College Board business
- Authorization to work in the United States
About Our Process
Application review will begin immediately and will continue until the position is filled
While the hiring process may vary, it generally includes: resume and application submission, recruiter phone/video screen, hiring manager interview, performance exercise, a panel interview, a conversation with leadership and reference checks.
About Our Benefits and Compensation
College Board offers a competitive benefits and compensation program that attracts top talent looking to make a difference in education. As a self-sustaining non-profit, we believe in compensating employees equitably in relation to each other, their qualifications, their impact, and the relevant market.
The hiring range for a new employee in this position is $92,000 to $150,000.
College Board differentiates salaries by location so where you live will narrow the portion of this range in which you can expect a salary. Your salary will be carefully determined based on your location, relevant experience, the external labor market, and the pay of College Board employees in similar roles. College Board strives to provide our best offer up front based on this criteria.
Your salary is only one part of all that College Board offers, including but not limited to:
- A comprehensive package designed to support the well-being of employees and their families and promote education. Our robust benefits package includes health, dental, and vision insurance, generous paid time off, paid parental leave, fertility benefits, pet insurance, tuition assistance, retirement benefits, and more
- Recognition of exceptional performance through annual bonuses, salary growth over time through market increases, and opportunities for merit raises and promotions based on increased scope of responsibility
- A job that matters, a team that cares, and a place to learn, innovate and thrive
You can expect to have transparent conversations about benefits and compensation with our recruiters throughout your application process.
About Our Culture
- We are motivated to positively impact the educational and career trajectories of millions of students a year
- We welcome staff to join any or all six of our affinity groups: ARISE (Alliance for Asian Retention, Inclusion, Success, and Engagement; DIASPORA (Alliance for Pan-African Success and Achievement); Pride (alliance for LGBTQ+ staff and allies); Resilience (alliance for Native staff and advocates); SALSA (Staff Alliance for Latinx Success and Achievement); and WIN (Women’s Impact Network)
- We value learning and growth; we offer formal and informal ways to lead through your superpowers, sharpen your strengths, and meet your development goals
- We know that our impact is strongest together. Our College Board Cares program offers all staff up to $1,000 annual match against partner non-profit organizations
- We offer a transparent approach to promotions and merit raises, annual performance-based bonuses, and how to grow your career here over time
#LI-TheAcord
#Li-Remote
