Evaluation Metric
An evaluation metric is one thing a person can be appraised on. Rejection rate. On-time dispatch. Teamwork. You build the client's list of them here, and the appraisal forms are then assembled out of that list.
What it is for
A measure answers one question about a person, and it answers it in one of exactly two ways.
Based on target. The measure produces a number, and the number is compared with a target. Rejection rate of 2.5 percent against a target of 2.0. Output of 48 tonnes a shift against 45. A measure on this basis has a unit and it has targets.
Score band. There is no number to compare. The evaluator answers on the one to five ladder, and the Evaluation Increment Matrix does the rest. Teamwork, discipline, initiative: anything a man is judged on rather than counted on. A measure on this basis has no target and no unit.
That choice is the only classifier on the screen. There is no answer type, no category and no objective type to set: one question replaced all three, and the one question is the scoring basis. See value lists.
There is also no rating scale screen anywhere in the module, and there does not need to be. The one to five ladder is intrinsic. What one to five is worth in money is the increment matrix's business.
The targets on this screen are seeds, not rules. A default Meet of 2.0 is what a new goal line starts at. The target that is actually scored is the one on the employee's own sheet, which is the whole point of agreeing targets person by person. Lock the target on the Evaluation Template when the company's figure must stand.
Direction comes from where the stretch target sits, not from a flag. Put the stretch above the target for a measure where more is better, and below it for one where less is better. Rejection rate with a target of 2.5 and a stretch of 1.0 is read correctly as smaller being better, with no other setting needed.
Every field, in plain words
The measure
| Field | What it means | What to put in it | Notes |
|---|---|---|---|
| Evaluation Metric | The short code for the measure | REJRATE | |
| Name | The measure as the employee reads it on his form | Rejection rate at final inspection | Required |
| Scoring Basis | How the measure is answered | Based on Target | Required. TGT Based on Target, a number against a target. BND Score Band, answered on the one to five ladder |
| Performance Unit, Name | What the number is counted in | PCT, Percent | Required for a Based on Target measure. Cleared and ignored for a Score Band one |
| Default Meet | The target a new goal line starts at | 2.5 | A seed. The sheet's own target is what is scored. Leave it at 0 if the target is always negotiated |
| Stretch | The figure that counts as full marks | 1.0 | Put it above Meet for bigger is better, below for smaller is better. It cannot equal Meet |
| Achievement Source | Who answers the measure | Manual | MAN Manual, a person types the achievement. SYS Computed, the system works it out. A computed measure is always Based on Target |
| Formula | The arithmetic that turns the figures below into one number | NETWEIGHT / SHIFTS | Only for a computed measure. Plain arithmetic over the aliases on the Measures tab, nothing else |
| Scope | Whose documents a computed measure reads | Own Documents | SLF Own Documents, what this man did. TEM Team, what the people under him did |
| Visibility | Who can see this measure exists | Public | PB Public, anybody. PV Private, HR only. SL Selected Employees, only the people listed on the Visibility tab |
| Notes | What the measure means and how it should be judged | Rejected pieces as a percent of pieces offered at final inspection. Excludes rework | Worth filling. It settles arguments later |
Key Results
Optional. The specific results a measure breaks down into, with a suggested weight each. They give the person writing a goal sheet something concrete to copy.
| Field | What it means | What to put in it | Notes |
|---|---|---|---|
| Line # | Line number | Fills itself | |
| Key Result | One result that makes up the measure | Final inspection rejection below 2 percent | |
| Default Weight | What this result is suggested to be worth | 50 |
Measures
Only for a computed measure. One line per figure the formula uses. Each line is given a short alias, and the formula is written in terms of those aliases.
| Field | What it means | What to put in it | Notes |
|---|---|---|---|
| Line # | Line number | Fills itself | |
| Alias | The short name the formula refers to this figure by | NETWEIGHT | Required. Letters, digits and underscore only, starting with a letter |
| KPI Source, Name | A saved source to take the document and its settings from | DISPATCH, Dispatch Notes | The easy way. Pick a source and most of this line fills itself |
| Screen | The document the figure is read from | Required if no KPI source is named | |
| Grid | The document's lines, when the figure is on a line | ||
| Aggregate | How the figures are combined | Sum | Required. SUM Sum. COUNT Count. AVG Average. MIN Minimum. MAX Maximum |
| Column, Column Name | The field holding the figure | Net weight | Required for every aggregate except Count |
| Date Column | Which date decides whether a document falls in the period | ||
| Employee Column, Employee Path | How the document points at the employee | Responsible to Employee | RSP responsible leads to the employee. USR the entering user leads to the employee. EMP the document carries the employee number |
| Filter Column | A field to narrow the documents by | ||
| Filter | How it is tested | Equals | EQ Equals. NE Not equal. GT Greater than. GE Greater or equal. LT Less than. LE Less or equal. IN In list. NEB Not blank. EMP Blank |
| Filter Value | What it is tested against | APPROVED | |
| Line Text | A note about this figure | Approved dispatches only |
Visibility
Only when Visibility is set to Selected Employees. One line per person who may see the measure.
| Field | What it means | What to put in it | Notes |
|---|---|---|---|
| Line # | Line number | Fills itself | |
| Employee | The person | 10427 | Required. Pick it from the list |
| Name, Department Name | His name and department | Fill themselves | |
| Reporting Level | How far down his reporting line the visibility also reaches | 1 for his direct reports |
Testing a computed measure
The screen can work the measure out for one person over one date range before anybody is appraised on it. Name a test employee and a date range you know has documents in it, run the test, and drill into the figures it used. Do this for every computed measure. A formula that looks right and returns zero is the most common setup fault in the module.
How to configure it
- Create the Performance Unit list first.
- Open the screen and name the measure the way the employee should read it. Rejection rate at final inspection, not REJ2.
- Choose the scoring basis. A number against a target is Based on Target. A judgement is Score Band.
- For a Based on Target measure, set the unit, a default Meet and a Stretch. Put the stretch on the correct side: 2.5 and 1.0 for rejection, 45 and 55 for output.
- Leave Achievement Source as Manual unless the figure genuinely exists in the system.
- For a computed measure, set it to Computed, add a Measures line per figure, give each an alias, and write the formula over those aliases. Set the scope to Own Documents or Team.
- Test the computed measure on a real employee over a real date range, and drill in to check the documents it picked up.
- Fill the Notes. Six months from now it is the only record of what the measure was supposed to mean.
- Save, then put the measure on the Evaluation Template sections that should carry it.
Scenarios
"Rejection rate should go down, not up"
Set Meet to 2.5 and Stretch to 1.0. Putting the stretch below the target is how the direction is recorded, and the scoring reads it that way round. There is no separate setting to find.
"We judge teamwork, we do not count it"
Score Band. No unit, no target, nothing to agree with the employee in advance. The evaluator answers on the one to five ladder and the Evaluation Increment Matrix decides what that is worth.
"The target is 45 tonnes a shift for everybody"
Put 45 as the default Meet here, then tick Lock Target on the Evaluation Template line. The company figure appears on every sheet and nobody can quietly reduce it.
"A manager should be measured on what his team dispatched, not on his own documents"
Same computed measure, scope set to Team. Keep the Own Documents version for the supervisors and the Team version for the managers, as two measures on two templates.
What it is connected to
- Performance Unit supplies the unit a Based on Target measure must have.
- KPI supplies a ready made source for a computed measure, so the document settings do not have to be typed line by line.
- Evaluation Template is assembled out of measures, and is where a target or a weight gets locked.
- Employee Goal carries the measure on a real person's sheet, with his own target and his achievement.
- Evaluation Tracker logs progress against a measure during the period.
- How a target and an achievement become a score is set out in the maths.
If something looks wrong
- "It will not save and asks for a scoring basis" - every measure is either Score Band or Based on Target. There is no third answer and no default.
- "It says the stretch cannot be the same as the target" - full marks and the target would be the same figure, so nothing could be scored between them. Move the stretch, above for bigger is better and below for smaller is better.
- "It asks for a unit and we do not want one" - the measure is Based on Target. If it is a judgement rather than a quantity, change it to Score Band and the unit requirement goes away.
- "It says a computed measure is always Based on Target" - it is. The engine produces a quantity, and a quantity is scored against a target.
- "The formula will not save and talks about an alias" - an alias has to be a plain word, letters digits and underscore, starting with a letter, and every name in the formula has to be one of the aliases on the Measures tab.
- "A computed measure returns zero" - test it. Either the date range catches no documents, or the employee path does not reach the person, or the filter excludes everything.
- "We cannot find the answer type or category field" - there is none. The scoring basis is the only classifier, and the old answer type, category and objective type fields are gone.