How AI Scoring Works

How Coco HR generates assessment questions, scores candidate answers, and what those scores mean — and don't mean.

Coco HR uses AI to do two things: generate relevant questions for a role, and evaluate the answers a candidate gives. This page explains how both work so that recruiters and integrators can interpret the results with the right level of confidence.

From skills to questions

When you create an assessment, you supply a list of required skills (for example, Python, React, AWS). Coco HR generates theory questions from these skills using a large language model, drawing on a curated skill catalogue (the same catalogue returned by GET /apikey/get_technologies). Using catalogue skill names verbatim produces the highest-quality questions; unrecognised names still work but yield weaker questions.

You control the shape of the assessment through the template:

Number of theory questions — how many AI-generated questions the candidate sees.
Per-question time limit — how long they have on each question.
Answer mode (audio or both) — whether answers are spoken, or the candidate can choose to type or speak.
Coding challenges — optionally enabled per skill, in addition to theory questions.

You can preview generated questions without committing to an assessment via POST /apikey/p-generate-questions.

Question kinds and how each is graded

Kind	How the candidate answers	How it's graded
Multiple choice (MCQ)	Picks one of four options.	Auto-graded — correct/incorrect, no AI judgement.
Theory (subjective)	Types or speaks a free-form answer.	Scored by an LLM against several dimensions.
Coding challenge	Writes and submits code.	The submission is scored by an LLM.

Spoken answers

When the answer mode is audio (or the candidate chooses to speak in both mode), the spoken response is transcribed using automatic speech recognition (Whisper / OpenAI) and then evaluated exactly like a typed answer. The transcript is stored in the report so a human can read what the candidate actually said.

What the AI scores

Theory answers

Each free-form theory answer is scored on three dimensions, each out of 5:

Dimension	What it measures
Completeness	Did the answer cover the key points?
Relevance	Was the answer on-topic for the question asked?
Technical	Was the technical content correct and substantive?

Coding submissions

Each coding submission is scored on:

Dimension	What it measures
Technical Proficiency	Quality and correctness of the code.
Problem Understanding	Did the candidate grasp what was being asked?
Solution Accuracy	Does the solution actually solve the problem?

Per-question and overall scores

Each question's dimensions roll up into a percentage shown with a badge:

Badge	Score
Meets Standard	≥ 60%
Partial	Below the "Meets Standard" threshold but non-trivial.
Inadequate	Low score.

The overall score is the average percentage across the candidate's questions. Reports also include written feedback per question so recruiters can see the reasoning behind a score, not just the number.

The AI model is configurable per company

By default, scoring runs on the platform's standard AI model. A company can instead bring its own model — OpenAI, Anthropic, OpenRouter, Groq, Mistral, or a custom endpoint. This matters because:

Scoring quality depends on the configured model. Different models reason differently, so scores are not guaranteed to be identical across providers.
If you change your provider, treat it as a change to your scoring pipeline — re-baseline what a "good" score looks like for your roles.

See Responsible AI and fairness for the implications of choosing your own model.

What a score means — and its limits

An AI score is decision support, not a decision. Keep these limits in mind:

There is no automatic hire / no-hire verdict from the AI. The model does not output "hire" or "reject." A human recruiter reviews the report and sets the candidate's verdict (Pending / Selected / Rejected).
Scores are estimates. They reflect how well an answer matched the model's judgement of a good answer for that question — not an absolute measure of ability.
Read the evidence. The report includes the actual answers, transcripts, per-question feedback, and recordings. Use them; don't rely on the headline number alone.
Context matters. A strong candidate can score lower on a poorly-worded spoken answer, or if transcription mangled a term. Reviewing the transcript catches this.

Human-in-the-loop

Coco HR is deliberately built so that AI assists screening rather than replacing the recruiter:

flowchart LR
    A[Candidate answers] --> B[AI transcribes & scores]
    B --> C[Report: scores, feedback, recordings, anti-cheat]
    C --> D[Human recruiter reviews]
    D --> E[Recruiter sets verdict:<br/>Pending / Selected / Rejected]

The verdict is always a human decision. AI narrows and structures the review; it does not make the call.

Anti-cheat and proctoring — the signals captured alongside scores.
Responsible AI and fairness — using scores fairly and lawfully.
Partner API reference — the get_candidate_report response shape.