The Center for Emerging Technology Identification for Roadway Infrastructure Construction & Maintenancehas developed a radar and rubric to evaluate emerging technologies for VTRC scientists to pilot and research.
Thisrubrichas undergone several iterations of refinementthroughdiscussions with facultyanditerations withLLMs.The existing version of the radar has been createdusing aretrievalaugmentedgeneration(RAG)concept. The LLMcompiles 30 URLs for each technology andusesthemto rate the technology against the rubric. These scores are then used to place the technology on the radar.Around 90 technologies have been rated; these wereidentifiedusinga VTRC Report on Emerging Technologies in 2004and research conducted by theCenter’spostdoctoral researchers.
As the Center shifts toward generating pilot projects for VTRC scientists,postdoctoral researchers,and undergraduate students to work on,there is still a need to verify the LLM outputandto builda more extensive database for each technology. The goal of this document is to outline a plan for undergraduate researchers hired by the Center to conduct this literature review andgenerate thesescoreswith thesupervision of the postdoctoral researchers.
Undergraduateresearchershired by the Center are expected to conduct the following tasks:
- Verifythe URLsprovided by the LLM. If these URLs are sufficient,noadditionalinformation is required.
- For URLs to be considered sufficient, they must meet the following criteria:Provide information about the performance of the technology
- Provide information about the cost of the technology
- Provide informationsuch that the technology can be compared with conventional practice
- Come fromcrediblesources(e.g.,.govor.edusites or official vendor sites)
If the URLs are not sufficient, the undergraduate researchers must conduct anadditionalliterature search using academic papers orthe websites ofindustryvendorsofferingthe technology.
- Generate a summary based on theliterature review. The summary should define the technologyand provide context forthe rubricratings. The summary shouldcontainreferences and be no longer than one page.
- Ratetechnologies based onthe literaturereview. The ratings should be self-supporting(justifiable from the summary alone). If ratings are based onjudgmentdue to alack of information, justification must be provided.
- Justification must also be provided onany rubricrowsthatwere omittedfrom scoring.
- Provide a confidenceflag for each technology (high/medium/low)to guideadditionalreview. If confidence is medium or low, postdoctoral researchers will conduct anadditionalliterature reviewbefore signing off on scoring.
One of the postdoctoral researchers will sign off on each technology before the score is committed to the radar. If the postdoctoral researcher disagrees with therating, thetwowill meetto comparethe reasoning behindeachscoreanddeterminewhetheradditionalinformationshouldbe added to the summary.
Undergraduate researchers willsubmitfor each technology,aone-pagePDFsummary along with arubric sheet highlighting the chosen scores and justification.A template will be provided for undergraduate researchers to fill out to ensure consistency.Both fileswill be stored in a folder named after each technology.
Undergraduate researchers mustbe able to synthesize information and provide summariesthat allow the Center team to assess each technologywithout referring to external sources.
Experience: Preference is for candidates with engineering backgrounds, but other adjacent fields will also be considered.
Undergrad student wage position for less than 20 hours per week.
Compensation: $16.50/hr.
MINIMUM REQUIREMENTS:
Resume required.
Education: Current UVA Undergraduate Student.
Experience: None
Licensure: None
Physical Demands: This position may require the ability to sit, stand, walk, bend, stoop, squat, climb stairs, use fine finger manipulation, etc., while conducting, observing, analyzing and/or recording research data.
Under the direct supervision of a faculty member or research supervisor,conducts research, follows protocols, records research results, etc.Will use various methods to collect, organize and analyze data, maintain records of research and/or experiments and document results. Ability to follow oral and written instructions, work effectively individually and in a team environment. Students are required to follow all policies and procedures required by the position, including but not limited to environmental, safety, information security/data privacy, etc.
The University of Virginia is an equal opportunity employer. All interested persons are encouraged to apply, including veterans and individuals with disabilities. Learn more about UVA’s commitment to non-discrimination and equal opportunity employment.
Data & Analytics (1)