Chapter 6. What We Assess

6. Scorecard, scales, weights and confidence

The scorecard is not about turning recruitment into arithmetic. Its purpose is to make candidate assessment reproducible: to pre-determine what matters for the role, how each criterion is tested, what assessment facts are considered sufficient and how confident the team is in the signal. A good scorecard protects the team from three common errors: each interviewer assesses different things, a strong impression replaces facts and the final decision is made before the assessment facts are discussed.

Universal scorecard template

Use one base format for all roles and adapt the criteria content to the role. In the template below, each row is a separate criterion that must be linked to a role task and tested by a specific method. If a criterion cannot be described through assessment facts, it cannot be used for decision-making.

Minimum completion rules:

1. The criterion must be observable. "Communication" is acceptable if there is a definition; "nice person" or "cultural fit" are not.

2. Type helps avoid mixing different kinds of signal.

Professional skill shows the ability to do the work; Soft skill shows work behaviour; Motivation shows alignment with role expectations; Signal reliability shows how much the evidence can be trusted.

3. Weight reflects the cost of getting the criterion wrong.

The more the criterion affects the first 3-6 months of results, the higher the weight.

4. The definition is written before the interview, not after meeting the candidate.

5. Mandatory assessment facts record what the interviewer must bring to the debrief.

Without assessment facts, the score remains an unverified impression.

6. The method must fit the criterion.

For example, discovery is best tested with a role-play scenario or case study, and reliability with behavioural interviews and references.

7. Rating 1 / 3 / 5 describes the scale anchors.

The interviewer must not invent the scale during the meeting.

Regulated / high-risk Responsibility, ethical behaviour, A mistake can create legal, finan-

reliability cial or safety risk

6. Scorecard, scales, weights and confidence

The scorecard is not about turning recruitment into arithmetic. Its purpose is to make candidate assessment

reproducible: to pre-determine what matters for the role, how each criterion is tested, what assessment facts are

considered sufficient and how confident the team is in the signal. A good scorecard protects the team from three

common errors: each interviewer assesses different things, a strong impression replaces facts and the final

decision is made before the assessment facts are discussed.

Universal scorecard template

Use one base format for all roles and adapt the criteria content to the role. In the template below, each row is

a separate criterion that must be linked to a role task and tested by a specific method. If a criterion cannot be

described through assessment facts, it cannot be used for decision-making.

Criterion Type Weight Definition Mandatory Method Rating 1 / 3 Confidence Notes

assessment / 5

facts

Criterion Professional 5-30% What the candidate Specific Structured 1 = insuffi- High / Questions, risks,

name skill / Soft must be able to facts, ex- interview, cient for Medium / contradictions,

skill / Motiva- do or demonstrate amples, work trial, role; 3 = work- Low and missing assess-

tion / Signal in the context of artefacts, case study, ing minimum; reason ment facts

reliability this role answers to portfolio 5 = strong

follow-up review, level for

questions, reference this role

work output, check,

candidate's role screening

Minimum completion rules:

1. The criterion must be observable. "Communication" is acceptable if there is a definition; "nice person"

or "cultural fit" are not.

2. Type helps avoid mixing different kinds of signal. Professional skill shows the ability to do the

work; Soft skill shows work behaviour; Motivation shows alignment with role expectations; Signal

reliability shows how much the evidence can be trusted.

3. Weight reflects the cost of getting the criterion wrong. The more the criterion affects the first 3-6

months of results, the higher the weight.

4. The definition is written before the interview, not after meeting the candidate.

5. Mandatory assessment facts record what the interviewer must bring to the debrief. Without assessment

facts, the score remains an unverified impression.

6. The method must fit the criterion. For example, discovery is best tested with a role-play scenario or

case study, and reliability with behavioural interviews and references.

7. Rating 1 / 3 / 5 describes the scale anchors. The interviewer must not invent the scale during the meeting.

8. Confidence is filled in separately from the rating. A high rating with low confidence means "it looks

like a strong signal, but there is little evidence."

9. Notes must not turn into a free-form essay. Write facts, short quotes, next-step questions and gaps.

Scale 1 / 3 / 5

Use practical scale anchors, not school grades:

Rating Meaning When to assign What should be in the assessment facts

1 Below working minimum or a risk Candidate does not demonstrate the right A concrete example of failure, absence

for the role behaviour, confuses the basic logic of the of the required action, a contradiction,

criterion, gives only generalities or shows a weak result, inability to explain

a red flag their contribution

3 Sufficient working level Candidate can perform the task in a A relevant work example, a clear

typical context but requires clear conditions, process, acceptable result, statement of

support or does not show consistency in constraints or level

challenging cases

5 Strong level for this role Candidate consistently demonstrates the Multiple relevant examples, complexity,

criterion under challenging conditions, independent role, measurable result,

explains trade-offs, sees risks and can mature explanation of decisions

improve the system around the task

If an intermediate rating of 2 or 4 is needed, use it only when the assessment facts fall between the scale anchors.

Do not turn the scale into "liked them a 4." An intermediate rating must be explainable through the neighbouring

scale anchors.

How to assign weights

Weights are not for pretty maths but for disciplined prioritisation. Without weights, the team often overvalues

bright but secondary signals: charisma, experience at a well-known company, a polished presentation or

resemblance to a successful employee.

Practical rules:

Situation Weighting Decision Example

Criterion is directly linked to the main business 20-30% For a CSM who must reduce churn, customer

result of the role risk diagnosis could carry 25%

Criterion is needed every day but is not the 10-20% Communication for most client-facing

primary differentiator roles

Criterion matters as a minimum threshold Pass / fail or 5-10% Ethical behaviour, legal eligibility,

basic literacy

Criterion can be developed after entry 5-10% Knowledge of a specific internal tool

without high risk

Criterion sounds desirable but is not linked Remove "Start-up mindset" if the role is

to role tasks actually process-driven and predictable

Recommended starting range for a role scorecard:

Type Typical total weight Comment

Professional skills 40-60% Tests the ability to produce a work outcome

Behavioural skills 25-40% Tests behaviour in an environment where

results are created with people

Motivation 10-20% Tests the durability of interest in the real role

Signal reliability Not always a weight, more often a confidence layer Does not replace criteria but shows how

much to trust the assessment facts

Weights must sum to 100% if the team uses a weighted view. But the final decision need not be an automatic

calculation: some criteria are threshold-based and may stop the process regardless of the average score.

Confidence rating

Confidence does not measure the candidate's quality but the quality of the signal. It answers the question:

"How confident are we that the assessment reflects real ability or risk?"

Confidence Meaning When to use

High Assessment facts are relevant to the role, Candidate broke down a similar case,

the next-step questions are specific, there are no explained their role, showed an artefact

major contradictions or completed a work trial

Medium Assessment facts are partially relevant but There is a good example but from a

there are constraints on context, depth or different context; details unverified; the

repeatability method was only an interview

Low Assessment facts are weak, indirect, Candidate spoke in generalities, interviewer

contradictory or obtained through an did not ask follow-up questions, no data on

inappropriate method a complex criterion

Record confidence as High - two relevant examples and a case, Medium - a similar example but

without metrics, Low - self-report only. A single word is not enough.

The difference between the score and confidence

The rating answers the question: "What level did the candidate demonstrate on this criterion?"

Confidence answers the question: "How reliably do we know that?"

Examples:

Situation Rating Confidence What to do

Candidate answers discovery questions 5 Medium Do not lower the rating automatically;

brilliantly but there is no role-play and add targeted verification or a

examples are only from small clients reference check

Candidate performed poorly on a work 1 High Treat as a strong negative assessment

trial for a key task; the method was close fact

to real work

When not to average scores. The mean is useful for review but dangerous as an automatic decision. Do not average ratings if:

1. There is a threshold criterion.

For example, ethical behaviour, legal eligibility, mandatory certification, safety, right to work with data.

2. The criteria tested different risk levels.

Weak customer risk diagnosis must not be offset by strong presentation if the role is responsible for churn.

3. Confidence differs significantly.

A single high rating with low confidence must not outweigh several medium high-confidence signals.

4. The methods are not comparable. A work trial on a real task is usually more reliable than a general conversation.

5. There is a red flag.

A red flag must be examined separately: confirm it, dismiss it or halt the process.

6. The role has mandatory constraint criteria.

For example, working hours, travel, language requirements, regulated access, conflict of interest.

7. The ratings reflect different stages.

A screening rating and a final case rating cannot be mechanically added together if they tested different criteria. Use the average only as a summary view after the debrief, not as a substitute for the decision. A proper debrief starts with the assessment facts per criterion, not with an overall candidate ranking. End-to-end example: minimal scorecards for BDM and Backend. Use these tables as a compact version of the scorecard. The full criteria library may be broader but the first working version is best kept short: 5-6 criteria, clear scale anchors and separate confidence.

1. There is a threshold criterion. For example, ethical behaviour, legal eligibility, mandatory

certification, safety, right to work with data.

2. The criteria tested different risk levels. Weak customer risk diagnosis must not be offset by

strong presentation if the role is responsible for churn.

3. Confidence differs significantly. A single high rating with low confidence must not outweigh

several medium high-confidence signals.

4. The methods are not comparable. A work trial on a real task is usually more reliable than a general conversation.

5. There is a red flag. A red flag must be examined separately: confirm it, dismiss it or halt the process.

6. The role has mandatory constraint criteria. For example, working hours, travel, language requirements,

regulated access, conflict of interest.

7. The ratings reflect different stages. A screening rating and a final case rating cannot be mechanically

added together if they tested different criteria.

Use the average only as a summary view after the debrief, not as a substitute for the decision. A proper debrief

starts with the assessment facts per criterion, not with an overall candidate ranking.

End-to-end example: minimal scorecards for BDM and Backend

Use these tables as a compact version of the scorecard. The full criteria library may be broader but the first

working version is best kept short: 5-6 criteria, clear scale anchors and separate confidence.

Criterion BDM Weight 1 3 5

B2B SaaS qualification High Speaks in Runs a clear Builds a

generality qualification repeatable

about sales logic process and

explains

trade-offs

Discovery quality High Presents before Asks baseline Identifies business /

understanding questions process / technical

the client and records blockers and

the pain the next step

Payment domain learning Medium Does not See baseline Connects

understand payment payment flow,

the payment constraints risk, integration

context and and buyer

does not know concerns

how to learn

Funnel ownership High Cannot explain Runs the Analyses win /

the stages / funnel and loss causes and

reasons next actions improves the

process

Stakeholder communication Medium Speaks the same Adapts language Holds complex

way to everyone to the role multi-stakeholder

Example scorecard: Customer Success Manager for B2B SaaS

Role context: The CSM manages B2B SaaS clients after the sale, owns adoption, renewal readiness, risk

visibility, escalation management and qualitative handover of context between sales, implementation, support and

product. The role is not a pure support position: the candidate must understand the client's business objectives,

spot risk before churn signals and communicate without making empty promises.

Criterion Type Weight Definition Mandatory Method Rating 1 / 3 Confidence Notes

assessment / 5

facts

Discovery Professional 20% Can surface Can identify Role-play, 1 = asks superficial Fill in post- Check that

skill business objectives, business goals, customer case, questions and interview discovery is

solution criteria, structured interview; immediately sells not replaced

stakeholders, interview the product

success metrics and with follow-up

implementation questions the candidate asks;

constraints how goal, pain, sets; how they record the

barriers and the next step

translates them into a plan

Client risk Professional 25% Early at detecting Real escalation Case interview, 1 = reacts only Fill in post- Key criterion

diagnosis skill churn risk, low or incident review behavioural after complaints interview for the role;

adoption, stakeholder breakdown or case; interview, or renewal risk; weak high-

loss, value gaps or list of risk signals; reference confidence

implementation drift actions in the first checks cannot be

The decision based on this scorecard must not be taken on a single average number. For this role, customer risk diagnosis is a near-threshold criterion: if the candidate scores 1 with high confidence, the process usually needs to be halted or a targeted check set only with very strong grounds. For motivation, a score of 3 is sufficient if professional skills are strong and role expectations have been discussed honestly. For communication and conflict maturity, it is important to look not only at the rating but also at the assessment facts: a calmer, less flashy candidate may be stronger than a candidate with polished but imprecise speech.

7. SOP: How to embed candidate assessment in the team

The SOPs in this section are designed as process documents that can be copied into a corporate knowledge base, ATS working guide or hiring operations manual and adapted to the role, stage, local legal requirements, internal policies and the specific ATS. They do not constitute legal advice. Before use in regulated hiring, mass recruitment, international processes or AI-assisted workflows, they must be reviewed with a legal or compliance lead. Each SOP below answers one operational question: who does what, when, on what inputs, which fields are mandatory, what counts as output and how to measure the quality of the process. If the company already has its own SOP format, transfer the content into it but do not remove the fields: Purpose, Scope, Roles, Inputs, Steps, SLA / timelines, Mandatory fields, Outputs, Exceptions, Metrics and Example. SOP: Professional Skills and Behavioural Skills

a score of 3 is sufficient if professional skills are strong and role expectations have been discussed honestly. For

communication and conflict maturity, it is important to look not only at the rating but also at the assessment

facts: a calmer, less flashy candidate may be stronger than a candidate with polished but imprecise speech.

7. SOP: How to embed candidate assessment in the team

The SOPs in this section are designed as process documents that can be copied into a corporate

knowledge base, ATS working guide or hiring operations manual and adapted to the role, stage, local

legal requirements, internal policies and the specific ATS. They do not constitute legal advice. Before

use in regulated hiring, mass recruitment, international processes or AI-assisted workflows, they must

be reviewed with a legal or compliance lead.

Each SOP below answers one operational question: who does what, when, on what inputs, which fields

are mandatory, what counts as output and how to measure the quality of the process. If the company

already has its own SOP format, transfer the content into it but do not remove the fields: Purpose,

Scope, Roles, Inputs, Steps, SLA / timelines, Mandatory fields, Outputs, Exceptions, Metrics and Example.

SOP: Professional Skills and Behavioural Skills

Field Process description

Purpose Define work-related Professional Skills and

Behavioural Skills for a specific role before

interviews begin, so that the team assesses the

ability to do the work rather than a CV, charisma

or personal preferences.

Scope Used for new roles, backfills, level changes,

launching a new hiring campaign and reviewing

the scorecard after a calibration meeting. Not

used for adjusting criteria to fit a favoured

candidate.

Roles The hiring manager owns the business outcomes

and must-have task criteria. The recruiter runs

the process and documents the criteria, checking

consistency across stages. Interviewers verify

whether the criteria are observable. Legal / HR

compliance reviews criteria where mandatory.

The role owner or hiring lead approves.

Inputs Draft job description, business outcome of the

role, success expectations for the first 3-6

months, key tasks, team context, failure modes

from past hires, legal or compliance constraints,

compensation / level range, existing competency

framework if available.

Steps 1. Start with the business outcome, not a skills

list. Record what the role must change or

protect in the business. 2. List 5-8 key tasks

where poor performance would be costly. 3.

For each task, define Professional Skills as

practical abilities, not tool names. 4. Define

Behavioural Skills only where they affect

performance in this environment. 5. Add

motivation risks that could lead to failure or

departure of a strong person.

6. Remove criteria that are not linked to work, not

observable or not testable.

7. Translate each criterion into scorecard fields: definition, assessment facts, method and scale

anchors 1 / 3 / 5.

8. Mark threshold criteria separately from weighted ones.

9. Check language for bias risk — for example cultural fit, executive presence, aggressive, native speaker, youthful

energy or similar labels.

10. Fix the criteria before the first interview with the candidate.

Example. For a Customer Success Manager in B2B SaaS, "knows SaaS" is too

broad. Convert it into Discovery, Client Risk Diagnosis,

Communication, Reliability, Conflict Maturity and Motivation for

Client Work. Each criterion receives a method and assessment

facts requirement before interviews begin.

SOP: Completing scorecards

Field Process description

Purpose Each interviewer records the rating by criterion, the facts and

confidence before the debrief, so that the hiring decision

rests on observable data rather than memory, status or group

influence.

Scope Applies to recruiter screens, hiring manager interviews, panel

interviews, technical and functional interviews, work trials,

case studies and reference checks, if they influence the

hiring decision.

Roles The interviewer completes the scorecard. The recruiter checks

completeness before the debrief. The hiring manager reads the

assessment facts, not just the recommendation on the CV.

The interview coordinator or ATS owner ensures the form is

available and linked to the right stage.

8. Flag contradictions and next-step questions explicitly. 9. Submit the scorecard before reading

other interviewers' opinions when the ATS allows it. 10. Do not change a submitted rating

after the debrief unless the change and reason are documented.

SLA / timelines Complete within 2 business hours after the interview and

always before the debrief. For same-day hiring processes,

complete within 30 minutes. Reference checks should be

documented before the final decision meeting.

Mandatory fields Candidate name or ID, role, stage, interviewer, date,

criterion, rating, notes with assessment facts, confidence,

missing assessment facts, risks, recommendation if

requested by the process, next-step questions,

submission time.

Outputs Completed scorecard in the ATS or shared hiring

document, clear assessment facts by criterion, list of

missing assessment facts, interviewer recommendation

with confidence level, next-step questions for the next

stage.

Exceptions If a criterion was not assessed, mark "not assessed" and

explain why; do not draw conclusions from an irrelevant

conversation. If the interview was interrupted, mark

confidence as Low and request a next step if the criterion

atters. If notes contain sensitive personal data unrelated to

the role, remove or escalate per company policy.

Metrics Scorecard completion rate before the debrief; percentage

of ratings with assessment facts; percentage of unverified

impressions moved to missing assessment facts; average

time from interview end to scorecard submission; number

of debrief delays caused by incomplete scorecards.

Example. An interviewer writes "works brilliantly with clients" after a

CSM interview. This cannot be used as an assessment fact. A

useful note reads: "In a risk case, identified sponsor change,

low adoption and missing executive goal; proposed a 48-hour

sponsor call, usage analysis and success plan. Rating 5,

confidence Medium because no reference check yet."

SOP: Debrief

Field Process description

Purpose Make the hiring stage decision by reviewing criteria and

assessment facts before opinions, resolving missing assessment

facts and assigning a candidate communication owner.

3. Review criteria before opinions: for each criterion, read the

rating, assessment facts and confidence. 4. Discuss the assessment

facts per criterion, not general impressions of the candidate. 5.

Identify missing assessment facts and decide whether this is

material. 6. Separate red flags from weak signals and unverified

impressions. 7. Compare contradictory ratings by examining the

method and assessment facts quality. 8. Choose one decision

option: advance, reject, pause, schedule a next step, make an

offer. 9. Assign the owner and deadline for candidate

communication. 10. Record the decision rationale in the ATS or

hiring document.

SLA / timelines The debrief is held within 1 working day after the last mandatory

interview for active candidates. Candidate communication must be

sent within the company's stated SLA for candidates, typically 1-2

working days after the decision.

Mandatory fields Candidate, role, stage, participants, criteria assessed, rating

and confidence summary, key assessment facts, missing

assessment facts, decision option, rationale, next action,

candidate communication owner, communication deadline.

Outputs Documented stage decision, candidate communication owner,

plan for the next stage or rejection rationale, next-action

assessment facts request if needed, updates to the ATS status.

Exceptions If scorecards are missing, postpone the debrief or explicitly note

the absent assessment facts; do not replace them with verbal

recollections. If a senior stakeholder states an opinion without

assessment facts, the facilitator asks which criterion and which

facts support it. If the legal team, compensation or access

constraints arise, pause the decision and route the question to

the appropriate owner.

Metrics Proportion of debriefs with completed scorecards before the

meeting; time from final interview to decision; percentage of

decisions with documented rationale; number of decisions

changed due to late assessment facts; candidate communication

SLA adherence.

Example. At the final CSM debrief, the team reviews Client Risk

Diagnosis before discussing "overall fit." One interviewer gave

a 5 from a case study, another gave a 3 from behavioural

responses. The team compares assessment facts, notes that the

case study is closer to real work and confidence is higher, then

advances with a next-step recommendation question on

reliability.

3. Review the scorecard criteria and scale anchors. 4. Compare

ratings and assessment facts by criterion. 5. Find where

interviewers used different definitions, different assessment fact

thresholds or unverified impressions. 6. Update the scale anchors

if the original wording is ambiguous. 7. Add training notes for

interviewers: common mistakes and more precise follow-up

questions. 8. Decide whether the stage or method needs

adjustment. 9. Record the changes and the effective date. 10.

Communicate the changes to all interviewers before the next

cycle.

SLA / timelines Monthly for stable hiring teams; fortnightly for active or

high-volume hiring teams; ad hoc within 5 working days after

a serious disagreement, process complaint or significant

quality miss.

Mandatory fields Meeting date, roles covered, attendees, candidate cases

reviewed, criteria discussed, rating differences, assessment

facts differences, scale anchor updates, interviewer training

notes, process changes, owner, deadline.

Outputs Updated rating scale anchors, clarified assessment facts

standards, interviewer training notes, action items for ATS or

interview plan changes, documented calibration history.

Exceptions Do not use calibration to retroactively adjust criteria to fit a

favoured candidate. Do not discuss protected or irrelevant

personal characteristics. If cases cannot be shared broadly,

use anonymised summaries or a smaller authorised group.

Metrics Interviewer rating variance by criterion; percentage of ratings

with sufficient assessment facts; pass-through rate by stage;

offer acceptance and pre-hire performance quality where

available; number of scale anchor updates; completion rate for

interviewer training notes.

2. Verify each claim against the raw notes. 3. Check alignment

with approved criteria: AI must not add new criteria after the

interview. 4. Cross-check weights and thresholds against the

scorecard. 5. Mark missing data without letting AI infer it. 6.

Check for bias risk and sensitive indicators. 7. Do not use AI

for hiring, rejection, level, compensation or final assessment

decisions. 8. If AI proposes a rating, treat it as a prompt for

human verification. 9. Log AI usage if company policy requires

it. 10. Escalate bias, privacy or automated-decision risk to the

process owner.

SLA / timelines AI review, if used, runs after human notes are recorded and

before the debrief. Human checks must be completed before

AI summaries are used in the decision discussion. Policy and

legal review must be completed before a new AI process is

launched.

Mandatory fields Tool name if mandatory, use case, source materials used,

human reviewer, criteria checked, unconfirmed AI statements

removed, missing assessment facts, bias risk check, final

human rating, decision owner, audit note if mandatory.

Outputs Human-validated assessment facts summary, cleaned

missing-assessment-facts list, next-step questions, AI usage

audit note where mandatory, rejected AI statements or

bias-risk notes.

8. Cases: How to handle complex situations

Case

1. Strong professional skills, weak behavioural skills

8. Cases: How to handle complex situations

Case 1. Strong professional skills, weak behavioural skills

Block Analysis

Situation The candidate confidently handles the technical / functional

part: solves the case, demonstrates deep expertise and

relevant results. However, in behavioural interviews, weak

signals appear in communication, responsibility or handling

conflict.

Risk The team may overvalue professional skills and hire someone

who creates rework, hides risks, damages trust or fails to

hold stakeholders. Another risk is rejecting a strong expert

due to general irritation, without separating trainable gaps

from critical work risks.

What to check Which behavioural skills are truly needed for this role; how

closely the weak signal links to job performance; whether the

gap can be offset by management, onboarding or allocation

of responsibilities; whether the behaviour is repeated across

multiple sources.

Questions / assessment facts "Tell me about a conflict with a colleague or client: what was

your contribution, what did you do, what did you change

afterwards?" "When was the last time you proactively flagged

bad news to a stakeholder?" Look for specifics, timelines,

ownership, consequences and the ability to learn.

Recommended action Do not decide on overall impression alone. Separate the

criteria: professional skills rating, behavioural skills rating,

confidence and impact on the role. If a soft skill is a threshold

criterion, add a next step or reference check. If the gap is

trainable, document the onboarding condition and the risk

owner.

How to document "Professional skill X is confirmed by assessment facts A/B

with high confidence. Soft skill Y has a weak signal: the

candidate did not mention early risk escalation in two

examples; impact on the role is high because the role requires

client escalation. A reference check on reliability is needed."