In short
Calibration software for consulting firms should do two things: support the calibration meeting with a distribution, a 9-box and logged changes, and estimate the leniency and severity of individual project leads statistically, with their uncertainty, as a proposal for the meeting. Many vendors document the meeting view, such as Leapsome, Lattice or Workday. Incompass and ValueScore from LEADBeyond document a statistical correction of raters.
Key takeaways
- Calibration takes two forms: the leaders’ joint meeting and a statistical estimate of how lenient or strict each rater is.
- In consulting firms, several project leads rate the same person; in 360° data, the raters’ own standards explained more variance than performance did.
- Good software shows rater effects with their uncertainty, proposes rather than changes automatically, and logs every adjustment with its reason.
- Statistics needs overlap: project leads must review people whom other leads review too.
- Most vendors document a meeting view with a 9-box or distribution; Incompass and ValueScore document a statistical correction.
- In Germany, criteria and procedure need the works council’s consent; we recommend that people decide.
Two kinds of calibration
People looking for calibration software usually mean one of two things. One kind supports the calibration meeting: leaders see the rating distribution, move people on a 9-box and record changes. The other kind calculates: it estimates from the data how lenient or strict each rater is and proposes a correction. Consulting firms need both, because the meeting sees context no statistic knows, and the statistics see patterns that get lost in the meeting.
Definition
Calibration software Software that makes performance ratings from several raters comparable: through a shared view for the calibration meeting, through a statistical estimate and correction of rater effects such as leniency and severity, or both. It prepares decisions on final rating, promotion and bonus.
| Meeting support | Statistical calibration | |
|---|---|---|
| What it does | Shows a distribution, 9-box or matrix; people move ratings | Estimates for each rater how far they sit above or below the others |
| Basis | Leaders’ judgement and evidence in the meeting | Overlap: the same people rated by several raters |
| Strength | Context, such as different tasks on different projects | Finds differences in standards that nobody voices |
| Limit | Whoever argues hardest tends to win | Needs enough overlap; the result is an estimate with uncertainty |
What is different in consulting firms
In a consulting firm, several project leads rate the same person every year. Each uses the scale in their own way: in two large samples of developmental 360° ratings, this idiosyncratic rater effect explained 62% and 53% of the variance, actual performance only 21% and 25% (Scullen, Mount and Goff, 2000). Whether someone worked for a lenient or a strict lead can therefore move their result more than their own work did.
The meeting alone does not fully solve this. At one large organisation, calibration committees adjusted 25% of ratings and curbed leniency, but made central tendency worse (Demeré, Sedatole and Woods, 2019). Software should therefore show not only the distribution but also which raters deviate systematically.
Seven requirements for calibration software
We derive this list from research on rater bias and from co-determination rules (LEADBeyond practice recommendation, not a measurement instrument).
- Raw data per rater and project. The software knows who rated whom on which project and how, not just the average.
- Estimate rater effects, do not just show distributions. A distribution shows that a team scores high; whether that reflects performance or leniency only shows when raters are compared on shared people.
- Show uncertainty. Every estimated effect comes with an interval. With few shared people it is wide, and that must be visible.
- Check overlap. The software reports which raters do not overlap with others and therefore cannot be corrected.
- Proposals, not automation. Corrections are proposals for the meeting; people decide and can depart from them with a reason.
- Log every change. Who changed what, when and why, traceable for the people rated, the works council and audits.
- A fixed handover to rating, promotion and bonus. The result follows from the calibrated score by a rule published in advance (Linking performance ratings to bonus).
What vendors document
Only statements from the vendors’ own pages (accessed 19 October 2026), with no prices and no ranking. “Not documented” means ask, not “not possible”. ValueScore is our own product; it sits in the same table as everyone else.
| Vendor | View for the meeting | Signals about raters | Statistical correction of leniency and severity |
|---|---|---|---|
| Leapsome | Heatmap, distribution and box view; changes logged | AI pre-reads flag inconsistencies such as leniency, change no rating | Not documented |
| Lattice | Adjust via table or 9-box | AI “Calibration Signals”: employees who may be over- or under-rated | Not documented |
| Workday | Distribution across a rating scale, drag and drop into boxes, target percentages | Not documented | Not documented |
| Personio | Rating distribution and 9-box, which its help centre calls useful during calibrations | Not documented | Not documented |
| Betterworks | Matrix from 3 to 25 boxes, changes in the table | Not documented | Not documented |
| 15Five | 9-box talent matrix, distribution by manager | Not documented | Not documented |
| Zensai Perform & Engage 365 | 9-box for talent and succession planning; no calibration feature documented | Not documented | Not documented |
| Confirm Perform | Team dashboard, admins adjust final ratings | Rules and AI flag ratings | Not documented |
| Incompass | 9-box and percentile distribution | Model per rater: leniency, severity, expertise | Yes: normalises results; model not published |
| ValueScore (LEADBeyond) | Calibration meeting on one page, level × rating matrix | Rater effects with a 95% confidence interval | Yes: leniency and severity are detected statistically and corrected |
Two observations. First, several vendors flag issues by AI or rules but leave the change to people; Leapsome states explicitly that its AI never changes a rating on its own. Second, only Incompass and ValueScore describe a model of the raters. Incompass addresses consulting firms as partners who deploy it for their clients; ValueScore is built for performance appraisals in consultancies and professional-services firms. 360° and survey tools such as Honestly or Crews & Captains document no calibration; Honestly protects anonymity through automatically checked minimum group sizes. Performance review software for consulting firms gives a broader comparison including pay and bonus.
How statistical calibration works
The principle is simple: every rating combines the person’s performance, the rater’s standard and noise. When several raters rate the same people, the standard can be estimated and removed, for example with least-squares models (Raymond and Viswesvaran, 1993) or Rasch models, which also detect central tendency, randomness and halo for each rater (Myford and Wolfe, 2003).
Works councils and data protection
- Statistics and decisions (EU). Recital 71 of the GDPR says the controller should use “appropriate mathematical or statistical procedures for the profiling”; Article 22 gives people the right not to be subject to a decision based solely on automated processing with legal or similarly significant effects. We recommend: the software proposes, people decide.
- Appraisal principles (Germany). Under § 94(2) of the Works Constitution Act (BetrVG), general appraisal principles need the works council’s consent, and the Federal Labour Court has extended this to the design of the appraisal procedure (BAG, 1 ABR 48/13). Criteria and anchors therefore belong in that conversation. That the calibration rules count too is our reading, not a ruling.
- Software (Germany). Software that is objectively suited to monitor performance can fall under § 87(1) no. 6 BetrVG (BAG, 1 ABR 7/15).
ValueScore for calibration
ValueScore is LEADBeyond’s appraisal platform for consultancies and professional-services firms: every appraisal, every project, every rater on one scale. Leniency and severity are detected statistically, shown with a 95% confidence interval and corrected. The calibration meeting fits on one page, as a level × rating matrix, and final rating, promotion and bonus are derived from normalised scores. We recommend treating every correction as a proposal that the meeting decides on.
Frequently asked questions
What is calibration software?
Software that makes performance ratings from several raters comparable. It supports the calibration meeting with a distribution, a 9-box and logged changes, ideally also estimates individual raters’ leniency and severity statistically, and so prepares decisions on final rating, promotion and bonus.
Which tools exist for calibrating feedback and reviews in consulting firms?
Leapsome, Lattice, Workday, Betterworks and 15Five, among others, document a meeting view with a distribution or 9-box; Personio shows a distribution and a 9-box that its help centre calls useful during calibrations. Leapsome, Lattice and Confirm add AI or rule-based signals. Incompass and ValueScore, our own product, document a statistical correction of leniency and severity. Per vendor pages, as of 19 October 2026.
What is the difference between a 9-box and statistical calibration?
A 9-box places people by performance and potential and helps the meeting align; the values in it come unchanged from the raters. Statistical calibration estimates how lenient or strict each rater is and removes that before the meeting. The two complement each other.
How many ratings do you need to estimate leniency or severity?
There is no fixed number. What matters is overlap: how many people a project lead rated who were also rated by others. With few shared people, the estimate is uncertain. Good software therefore shows an interval and reports whom it cannot correct.
Do you still need a calibration meeting if software corrects statistically?
Yes. Statistics cannot see whether a project lead rates more strictly or ran harder projects. The meeting checks such reasons, decides on every correction and records departures. Our recommendation: statistics before the meeting, decisions in it.
May calibration software change ratings automatically?
Article 22 GDPR gives people the right not to be subject to a decision based solely on automated processing with legal or similarly significant effects. We recommend that the software proposes corrections and people decide. Leapsome, for example, states that its AI never changes a rating on its own. Not legal advice.
Does a works council have to agree to calibration software?
In Germany, this needs checking. General appraisal principles need the works council’s consent under § 94(2) BetrVG, and the Federal Labour Court has extended this to the design of the appraisal procedure. In our reading, the calibration rules arguably fall under it too (not a ruling). Software objectively suited to monitoring performance can fall under § 87(1) no. 6 BetrVG. Involve the works council early. Other countries have their own rules. Not legal advice.
Is 360° feedback calibrated?
We recommend separating two purposes. 360° feedback for development depends on anonymity and conversation; there, minimum group sizes protect the raters. Calibration applies to reviews that lead to rating, promotion or bonus, such as a consulting firm’s project reviews.
Sources
- Scullen, S. E., Mount, M. K., & Goff, M. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85(6), 956–970., American Psychological Association (2000). Peer-reviewed study of two large samples of 360° ratings (2,350 and 2,142 managers, seven ratings each). Cited for the shares of variance given in the abstract: idiosyncratic rater effects 62% and 53%, actual performance 21% and 25%. The data are developmental ratings; they do not establish shares for pay-linked appraisals or for consulting firms.
- Demeré, B. W., Sedatole, K. L., & Woods, A. (2019). The role of calibration committees in subjective performance evaluation systems. Management Science, 65(4), 1562–1585., INFORMS (2019). Peer-reviewed field study using internal data from a large multinational organisation. Cited (from the abstract) for the 25% adjustment rate, less leniency and more central tendency. One organisation, so transfer is open.
- Raymond, M. R., & Viswesvaran, C. (1993). Least squares models to correct for rater effects in performance assessment. Journal of Educational Measurement, 30(3), 253–268., Wiley (National Council on Measurement in Education) (1993). Cited for correcting leniency and severity when raters overlap, and for the medical oral exam example (40 examiners, about 115 candidates, four examiners each) in which adjustment would have changed pass or fail for about 7–9% of candidates in a reanalysis. Applying this to business appraisals is our inference.
- Myford, C. M., & Wolfe, E. W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4(4), 386–422., JAM Press (2003). Methods paper without a DOI (abstract via PubMed); cited for detecting, per rater, leniency or severity, central tendency, randomness and halo. Part II: 2004 (Journal of Applied Measurement, 5(2), 189–227). Not evidence that a typical firm’s data have enough overlap for it.
- Running a calibration meeting – Leapsome Help Center, Leapsome GmbH (2026). Vendor page, accessed 19 October 2026: calibration via heatmap, distribution and box view; rating changes are logged; the reviewee sees only the calibrated score.
- Performance Reviews – Leapsome, Leapsome GmbH (2026). Vendor page, accessed 19 October 2026: project-based reviews; AI pre-reads flag inconsistencies such as manager leniency, and “the AI never changes a rating on its own”.
- Calibrate ratings in a talent review – Lattice Help Center, Lattice (2026). Vendor page, accessed 19 October 2026: scores adjusted via a table or 9-box view.
- Lattice Fall/Winter 2026 product release, Lattice (2026). Vendor page, accessed 19 October 2026: AI “Calibration Signals” surface employees who may be over- or under-rated relative to their performance signals.
- Overview of cycle metrics – Personio Help Center, Personio SE & Co. KG (2026). Vendor page, accessed 19 October 2026: distribution of performance and potential ratings and a 9-box matrix, “useful … during calibrations”.
- Talent and Performance Calibration – Workday Administrator Guide, Workday (2026). Vendor page, accessed 19 October 2026: calibration distributes employees across a rating scale; drag and drop into boxes, target percentages.
- Calibration – Betterworks Support, Betterworks (2026). Vendor page, accessed 19 October 2026: matrix from 3 to 25 boxes with changes in the table; additional contributor responses are not passed to calibration; field for a recommended raise.
- Calibrations in performance reviews overview – 15Five, 15Five (2026). Vendor page, accessed 19 October 2026: 9-box talent matrix of manager ratings; manual adjustment.
- Succession planning software – Zensai, Zensai (2026). Vendor page, accessed 19 October 2026: 9-box grid for performance and potential in succession planning; no calibration feature described.
- Calibration: how to manage the team dashboard during cycles – Confirm Support, Confirm (2026). Vendor page, accessed 19 October 2026: rules flag ratings that need attention; admins adjust final ratings.
- Perform – Confirm, Confirm (2026). Vendor page, accessed 19 October 2026: AI flags when managers may be rating on visibility rather than performance.
- Performance Calibration – Incompass, Incompass Labs, Inc. (2026). Vendor page, accessed 19 October 2026: models each reviewer’s leniency, severity and expertise and normalises results; 9-box and percentile distribution.
- Partners – Incompass, Incompass Labs, Inc. (2026). Vendor page, accessed 19 October 2026: management consulting firms as partners who deploy Incompass for their clients.
- 360°-Feedback – Honestly, Honestly MT GmbH (2026). Vendor page, accessed 19 October 2026: 360° feedback comparing self-view and others’ view, with automatic minimum group sizes. In German.
- 360-Grad-Feedback-Beratung – Crews & Captains, CREWS & CAPTAINS GmbH (2026). Vendor page, accessed 19 October 2026: 360° feedback with its own software, managed service and result reports. In German.
- § 94 BetrVG – Personalfragebogen, Beurteilungsgrundsätze (Works Constitution Act, personnel questionnaires and appraisal principles), gesetze-im-internet.de (German Federal Ministry of Justice) (2026). Official German text as of the research date, 14 October 2026. Cited for para. 2 (works council consent to general appraisal principles); the English rendering is ours. Not legal advice.
- § 87 BetrVG – Mitbestimmungsrechte (Works Constitution Act, co-determination rights), gesetze-im-internet.de (German Federal Ministry of Justice) (2026). Official German text as of the research date, 14 October 2026. Cited for para. 1 no. 6 (technical devices for monitoring behaviour or performance); the English rendering is ours. Not legal advice.
- Regulation (EU) 2016/679 (General Data Protection Regulation) – Article 4(4), Article 22 and Recital 71, EUR-Lex, Publications Office of the European Union (2016). Official text. Cited for the definition of profiling (performance at work), the right not to be subject to a decision based solely on automated processing with significant effects, and Recital 71’s “appropriate mathematical or statistical procedures”. Not legal advice; UK GDPR is not covered.
- Bundesarbeitsgericht (Federal Labour Court), decision of 17 March 2015 – 1 ABR 48/13, Bundesarbeitsgericht (2015). Cited for the headnote: co-determination under section 94(2) BetrVG extends to the design of the appraisal procedure. The case concerned appraisal conversations, not calibration; that calibration rules fall under it is our reading. Not legal advice.
- Bundesarbeitsgericht (Federal Labour Court), decision of 13 December 2016 – 1 ABR 7/15, Bundesarbeitsgericht (2016). Cited for para. 22: technical devices are intended for monitoring if they are objectively suitable for it. The case concerned a Facebook page; applying it to calibration software is our reading. Not legal advice.