Generalist - Language AI Evaluation

New

Skills

Ai Analytical Thinking Content Writing Data Annotation Model Evaluation Natural Language Processing

Mercor partners with leading AI teams to evaluate and improve general-purpose conversational AI systems. This project focuses on evaluating and improving general chat behavior in large language models (LLMs). The role involves evaluating model-generated responses, conducting fact-checking, and providing high-quality feedback to ensure AI systems communicate effectively.

Key Responsibilities
  • Evaluate LLM-generated responses
  • Conduct fact-checking using trusted sources
  • Generate high-quality human evaluation data
  • Assess reasoning quality, clarity, and completeness of responses
  • Ensure model responses align with expected conversational behavior
Requirements & Qualifications
  • Bachelor's degree required
  • Native speaker or primary fluency in Italian
  • Experience with large language models
  • Excellent writing skills
  • Strong attention to detail
  • Adaptable across topics and domains
  • Background in structured analytical thinking
  • College-level mathematics skills

No forms. Your profile is generated instantly.

Job Type: Remote

Salary: Not Disclosed

Experience: Entry

Duration: 12.0 Months

Share this job:

Similar Jobs

Remote AI Analyst

Posted 54 days ago

Drive customer experience automation

Optimize business processes with AI solutions

Ai Business Analyst Data Analysis Data Visualization

Principal AI Engineer Role

Posted 54 days ago

Hire a remote Principal AI Engineer

Develop customer experience automation solutions

Ai Automation AWS Cloud Computing

Staff Software Engineer

Posted 54 days ago

Revolutionize enterprise data operations through AI solutions.

Automate and accelerate data tasks for overworked data teams.

Ai Airflow Ansible Api Development

Latin America Remote Opportunities

Posted 54 days ago

Find remote job opportunities, Explore work-from-home options, Expand career in Latin America, Gain

global experience, Work with international

Content Writing Customer Service Data Analysis Digital Marketing

AI Trainer for Gaming

Posted 54 days ago

Improve large language model accuracy

Support immersive game content creation

Collaboration Communication Data Analysis Data Annotation

AI Trainer for Games

Posted 54 days ago

Enhance AI model accuracy and performance

Annotate and evaluate in-game content

Content Development Data Annotation English Fluency Remote Collaboration

AI Trainer Role

Posted 54 days ago

Enhance AI dialogue quality

Provide high-quality training data

Ai Ai training Collaboration Communication

Senior Product Manager, AI Agents

Posted 54 days ago

Lead product development and strategy for AI Agents

Collaborate with cross-functional teams

Ai Analytical Skills Communication Skills Cross-functional Collaboration

Staff Product Designer, AI

Posted 54 days ago

Design and implement AI solutions within the Apollo platform.

Lead end-to-end design processes for product development.

Adobe Creative Suite Ai API Design Collaboration Tools

Senior AI Engineer

Posted 54 days ago

Build and productionize advanced AI systems

Develop large language model platforms

Ab testing Ai Systems Api Integration Architecture

Senior AI Engineer Role

Posted 54 days ago

Build and deploy scalable AI systems for production use.

Develop advanced multi-agent architectures and conversational AI.

Api Integration Architecture AWS Backend Development

AI Product Designer Role

Posted 54 days ago

Hire a remote AI product designer

Enhance workplace learning platforms

Adobe Creative Suite Agile Ai Collaboration Tools

Senior AI Media Manager

Posted 54 days ago

Recruit for a remote senior product manager role

Lead AI-driven media product development

Agile Methodologies Ai Data Analysis Market Research

AI Team Engineering Lead

Posted 54 days ago

. Lead and manage the AI Team effectively

. Drive innovation in AI technologies

Big Data Cloud Computing Data Science Java

Senior AI Product Manager

Posted 54 days ago

Drive AI quality in collaboration products

Lead remote cross-functional teams

Agile Methodologies Ai Data Analysis Machine Learning

AI Senior Design Manager

Posted 54 days ago

Lead AI-focused design initiatives

Manage and mentor remote design teams

Ai Collaboration Tools Cross-functional Collaboration Machine Learning

Experienced Software Engineer at Automattic

Posted 54 days ago

Seeking experienced software engineers for global projects

Emphasizing a positive and inclusive work culture

Ai Continuous integration Engineer Gutenberg

Senior Product Designer

Posted 54 days ago

Design exceptional products from start to finish.

Iterate actively with prototypes.

Ai Communication Communication Skills Design Systems

Experienced Software Engineer

Posted 54 days ago

Seeking experienced software engineers globally

Focused on open-source contributions

Accessibility Ai Ai technology Architecture

Sr. Principal - HRIS

Posted 54 days ago

Lead HR transformation through Workday optimization

Guide cross-functional teams in complex Workday changes

Ai Automation Data Analytics Documentation

Senior DevOps Engineer

Posted 54 days ago

Build automation processes and best practices in software development.

Contribute to system efficiencies through reusable automation scripts and tools.

Ai Architecture Automated Testing AWS

AI Sales Specialist Role

Posted 54 days ago

Drive adoption of AI-powered solutions

Support and enable sales strategies

Analytics Communication Machine Learning Natural Language Processing

Partner Technical Account Manager

Posted 54 days ago

Expand partner program participation and integrations

Act as primary technical liaison for partners

Account manager Analytical Thinking Cloud Cloud Technologies

Head of Marketing

Posted 54 days ago

Hiring a remote Head of Marketing with extensive experience in Meta & Google Ads and email marketing

Seeking a data-driven individual to optimize digital ad campaigns and email marketing strategies

A/b Testing Analytical Thinking Autonomy Communication Skills

QA Engineer Fusion Project

Posted 54 days ago

Ensure high-quality product delivery

Collaborate with cross-functional teams

Analytical Thinking Engineer Jira QA

Sr. Product Manager Role

Posted 54 days ago

Recruit a senior product manager

Lead Duo Agent Platform strategy

Agile Methodologies Ai Cloud Platforms Data Analysis

Senior Product Designer, AI at GitLab

Posted 54 days ago

Hiring a Senior Product Designer, AI remotely

Full-time position in Americas or EMEA

Ai Ai Design Collaboration Figma

AI Research Manager

Posted 54 days ago

Lead research direction for advanced AI systems

Guide the design of cutting-edge RAG systems

Data Analysis Deep Learning Documentation Leadership

Stealth AI Product Manager

Posted 54 days ago

Drive zero-to-one development of a stealth AI product

Own end-to-end product experience for an AI-powered tool

Ai Cross-platform Development Onboarding Product Development

AI Data Specialist Role

Posted 54 days ago

Transform education and productivity through AI

Ensure high-quality labeled data for model training

Analyst Collaboration Tools Communication Skills Data Annotation

AI Data Specialist

Posted 54 days ago

Generate accurately labeled data for AI models.

Support model training and evaluation.

Collaboration Tools Communication Skills Data Annotation Data Labeling

Principal AI Engineer

Posted 54 days ago

Design, develop, and deploy AI solutions

Drive project lifecycle from conception to deployment

AWS Cloud Platforms Docker Engineer

Lead UX Designer

Posted 54 days ago

Seeking experienced UX Designer for complex applications

Specializing in Supply Chain, Data Management, and AI initiatives

Ai Collaboration Communication Css

Trading Insights Manager

Posted 54 days ago

Design and develop trading insight products, Utilize big data and AI for actionable market

ties, Collaborate with cross-functional teams, Analyze product performance and user feedback,

Ai Big Data Data Analysis English

Remote AI Product Manager

Posted 54 days ago

Recruit an AI Product Manager

Enable remote full-time work

Agile Methodologies Agile Methodology Ai Data Analysis

Remote Support Specialist Writer

Posted 54 days ago

Hire a remote support specialist

Recruit a skilled writer for support content

Content Writing Customer Support Editing Email Communication

Senior Software Engineer, Global Contractor

Posted 54 days ago

Empower financial advisors and clients through AI technology

Develop smarter workflows and enterprise-grade security solutions

Ai Ai technology ChatGPT Engineer

Global Remote Software Engineer

Posted 54 days ago

Empower financial advisors with AI

Hire skilled global software engineers

Ai Ai Tools Api Integration Engineer

iOS Engineer Swift UIKit SwiftUI

Posted 54 days ago

Build and ship high-quality iOS features and apps

Implement modern iOS UI with UIKit and SwiftUI

Ai Automated Testing CI/CD Engineer

Senior Digital Copywriter Role

Posted 54 days ago

Craft high-converting digital ad copy

Optimize content for PPC and landing pages

Advertising Content Strategy Content Writing Conversion Rate Optimization

Director - Labs (Lean Startup)

Posted 54 days ago

Incubate and scale new AI-driven business opportunities

Identify, prioritize, and evaluate new ideas

Agile Ai Computer science Data-driven decision-making

AI Engineer Fixed-Term

Posted 54 days ago

Hire remote AI Engineer

Develop educational AI solutions

Ai Cloud Computing Data Analysis Engineer

Alignerr Support Specialist

Posted 54 days ago

Manage inbound support queries efficiently

Resolve complex cases and escalate when needed

Ai Communication Communications Communication Skills

Senior AI/Python Engineer

Posted 54 days ago

Seeking Senior Engineer with extensive experience in AI/Python

Remote job opportunity for skilled professionals

Ai Architecture AWS Cloud Computing

Senior Golang Developer

Posted 54 days ago

Looking for Senior Golang Developers with strong technical skills in AI and Kubernetes

Remote job opportunity requiring advanced English communication skills

Ai English Go Kubernetes

AI Product Marketing Manager

Posted 54 days ago

Communicate product value and differentiation

Lead strategic marketing for AI solutions

Competitive Analysis Content Creation Customer Experience Enterprise Software

Customer Success Manager

Posted 54 days ago

Develop and manage engagement plans, provide tailored solutions, analyze data for insights, drive

stomer engagement and adoption, contribute to operational

Ai Crm Front Project Management