Data Model Quality Analyst Job at Russell Tobin, San Jose, CA

U3M5TWpZWFBzOFBsNStXUjdVb2NwMjlYZkE9PQ==
  • Russell Tobin
  • San Jose, CA

Job Description

Russell Tobin & Associates is currently seeking a Hybrid Data Model Quality Analyst to work for our client in the AI/tech industry. Apply now for consideration!

Contract: W2, 6- Months

Location: Hybrid | Tues–Thurs On-site in San Jose, CA

Pay Range: $75-85/Hour depending on experience

Responsibilities:

  • Ensure data quality during task development for human-created content used in training Large Language Models (LLMs).
  • Define detailed data requirements based on model goals and analysis of model loss patterns.
  • Design high-quality, nuanced instructions for expert content creators with illustrative examples.
  • Rapidly iterate with content-production vendors to improve data quality in small batch reviews.
  • Develop benchmark datasets to assess and validate data quality.
  • Benchmark vendor outputs against competitor models using both quantitative and qualitative methods.
  • Analyze data from vendor teams and optimize rater pools for quality.
  • Audit datasets, identify quality issues, and help develop tools to streamline data review processes.
  • Use experimentation to demonstrate data impact and uncover model blind spots.
  • Generate regular stakeholder reports on data quality metrics, iteration times, and vendor performance.
  • Author and maintain best practices documentation and instructional guidelines.

Requirements:

  • Bachelor’s degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.
  • 3+ years of experience in Technical Writing, Quality Assurance, or a related field.
  • Excellent writing and editing skills with adaptability to varied styles and tones.
  • Strong analytical and problem-solving skills.
  • Ability to work independently and collaboratively across technical and non-technical teams.

Preferred Qualifications:

  • PhD in a relevant field (English, Linguistics, Computer Science, etc.)
  • 5+ years experience in relevant roles with exposure to LLM development or content creation at scale.
  • Experience with data benchmarking, quality auditing, or content vendor management.
  • Familiarity with model evaluation techniques and qualitative dataset analysis.

Russell Tobin / Pride Global offers eligible employees comprehensive healthcare coverage (medical, dental, and vision plans), supplemental coverage (accident insurance, critical illness insurance, and hospital indemnity), a 401(k)-retirement savings plan, life & disability insurance, an employee assistance program, identity theft protection, legal support, auto and home insurance, pet insurance, and employee discounts with preferred vendors.

#LI-JJ2

Job Tags

Contract work,

Similar Jobs

HCRC Staffing

Chiropractor Manhattan NY Job at HCRC Staffing

 ..., greater flexibility, and improved range of motion. In addition to chiropractic care, patients can receive soft-tissue massage, acupuncture, and physical therapy. We pride ourselves on providing personalized, compassionate care in a welcoming environment that is easily... 

Beth Israel Lahey Health

Patient Transporter Job at Beth Israel Lahey Health

 ..., youre making a difference in peoples lives. Contributes to the Hospitals mission by supporting our clinical teams in transporting our patients to various departments during their stay and upon discharge. This clinical support staff position will have an impact on safety... 

HCRC Staffing

Chiropractor Thatcher AZ Job at HCRC Staffing

 ...amazing team. If this sounds like you, then we want you and your unique skills and goals to join our dynamic team! Certification in acupuncture would be helpful but not necessary. About us: We are a Chiropractic clinic that provides treatments for a wide range of... 

Eastern Therapeutics

Acupuncturist Job at Eastern Therapeutics

 ...A well established Acupuncture/ Chinese herbal practice in Walpole, MA is seeking a Part-Time Acupuncturist to start ASAP. The shift is 2-3 days /wk (~45-70 patients). We currently have a 1-2wk wait list for an appointment at our clinic. Since 2005, our clinic has earned... 

Ginas Tech Jobs

Product Manager, Java and Android App Experience Required - Work From Home Job at Ginas Tech Jobs

 ...Product Manager, Java and Android App Experience Required Work From Home These Product Manager opportunities are 100% Remote...  ...the stakeholders to resolve any conflicting requirements and/ or development priorities. Collaborating with the team to clarify...