Data Model Quality Analyst Job at Russell Tobin, San Jose, CA

U3M5TWpZWFBzOFBsNStXUjdVb2NwMjlYZkE9PQ==
  • Russell Tobin
  • San Jose, CA

Job Description

Russell Tobin & Associates is currently seeking a Hybrid Data Model Quality Analyst to work for our client in the AI/tech industry. Apply now for consideration!

Contract: W2, 6- Months

Location: Hybrid | Tues–Thurs On-site in San Jose, CA

Pay Range: $75-85/Hour depending on experience

Responsibilities:

  • Ensure data quality during task development for human-created content used in training Large Language Models (LLMs).
  • Define detailed data requirements based on model goals and analysis of model loss patterns.
  • Design high-quality, nuanced instructions for expert content creators with illustrative examples.
  • Rapidly iterate with content-production vendors to improve data quality in small batch reviews.
  • Develop benchmark datasets to assess and validate data quality.
  • Benchmark vendor outputs against competitor models using both quantitative and qualitative methods.
  • Analyze data from vendor teams and optimize rater pools for quality.
  • Audit datasets, identify quality issues, and help develop tools to streamline data review processes.
  • Use experimentation to demonstrate data impact and uncover model blind spots.
  • Generate regular stakeholder reports on data quality metrics, iteration times, and vendor performance.
  • Author and maintain best practices documentation and instructional guidelines.

Requirements:

  • Bachelor’s degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.
  • 3+ years of experience in Technical Writing, Quality Assurance, or a related field.
  • Excellent writing and editing skills with adaptability to varied styles and tones.
  • Strong analytical and problem-solving skills.
  • Ability to work independently and collaboratively across technical and non-technical teams.

Preferred Qualifications:

  • PhD in a relevant field (English, Linguistics, Computer Science, etc.)
  • 5+ years experience in relevant roles with exposure to LLM development or content creation at scale.
  • Experience with data benchmarking, quality auditing, or content vendor management.
  • Familiarity with model evaluation techniques and qualitative dataset analysis.

Russell Tobin / Pride Global offers eligible employees comprehensive healthcare coverage (medical, dental, and vision plans), supplemental coverage (accident insurance, critical illness insurance, and hospital indemnity), a 401(k)-retirement savings plan, life & disability insurance, an employee assistance program, identity theft protection, legal support, auto and home insurance, pet insurance, and employee discounts with preferred vendors.

#LI-JJ2

Job Tags

Contract work,

Similar Jobs

Capstone Logistics, LLC

Warehouse Operations Manager Job at Capstone Logistics, LLC

 ...solutions partner with over 650 operating locations, 19,000 associates, and 60,000 carriers. We specialize in freight management, warehouse and distribution center support, last-mile delivery, supply chain analytics, optimization, and more. Our integrated, end-to-end logistics... 

Veyo

PART TIME DRIVER GIG | MORNINGS | AFTERNOONS Job at Veyo

Have a car? Earn a $1,000 BONUS + $10,000 GUARANTEE* when you sign up to drive your own vehicle and help transport others to healthcare appointments around Phoenix! Veyo is a mission-driven company helping individuals get to and from their non-emergency medical appointments...

VPNE

General Manager - Hotel Valet Operations (Sarasota) Job at VPNE

 ...General Manager - Hotel Valet Operations Join to apply for the General Manager - Hotel Valet Operations role at VPNE . The role of the Valet & Parking General Manager is to oversee and manage the hotel valet operation, parking, and lot services by providing exceptional... 

In House Jobs | JDHuntr

In House Counsel Jobs New York | JDHuntr 37298 Associate, Criminal Justice Reform Program, Litigatio Job at In House Jobs | JDHuntr

In House Counsel Jobs New York | JDHuntr 37298 Associate, Criminal Justice Reform Program, Litigation, San Francisco, CA\ To apply go to JDHuntr.com \ Assisting on grantee check-in calls and helping the team assess how grants are going. Attending convenings, strategy... 

Exos

Member Services Lead Job at Exos

 ...Flexibility: able to work evenings, weekends and holidays WHO WE ARE For nearly 30 years, Exos has been dedicated to getting people ready for the moments that matter by promoting a holistic approach to health. As an elite coaching company, Exos applies its evidence-...