Comparing One Firm With Another
Published benchmarks compare numbers built from unpublished definitions. What survives the comparison and what does not.
Industry benchmarks for utilisation are widely published and widely cited. They compare figures whose definitions are not published, which makes the comparison weaker than it appears.
The practical lesson in “Comparing One Firm With Another” is that a convenient number should open a question rather than end one. A team considering review the available configuration choices for employee monitoring software with screenshots can add consistent time and project context, with transparent settings, limited access and a documented review process around every material interpretation.
This is not an argument that benchmarks are useless. It is an argument about which conclusions they can carry.
As a separate benchmark for “Comparing One Firm With Another”, see IRS recordkeeping guidance. External guidance is most useful when it prompts a specific check rather than replacing case-by-case judgement.
The definition problem
A survey asks firms for their utilisation figure. Each respondent answers using their own denominator, their own exclusion list and their own treatment of write-offs.
- Denominator — contracted hours, a standard year, or available hours.
- Exclusions — leave, holidays, committed internal time, ramp-up.
- Write-offs — counted in the numerator or removed from it.
- Population — fee earners only, or everybody who records time.
The resulting average is a mean of numbers measuring different things, reported to one decimal place.
The selection problem
Firms with embarrassing figures answer surveys less often than firms with good ones. The direction of the bias is known and its size is not.
Benchmarks drawn from voluntary response are therefore upper estimates of the population, by some unknown amount.
The mix problem
Utilisation depends heavily on seniority mix, commercial model and practice area. A firm with many juniors on time-and-materials work will report higher than one of equal health with a senior advisory mix.
Comparing against an average across mixes compares against a firm that does not exist.
What a benchmark can legitimately do
Tell you the rough order of magnitude. If a firm reports fifty per cent against a sector that clusters around seventy-five, something is worth investigating — a definition difference, most likely, or a genuine problem.
Large gaps are informative. Gaps of a few points are noise.
What it cannot do
Set a target. A target drawn from a benchmark imports another firm's definition, mix and model into your incentive system.
That is how firms end up chasing a figure that was never achievable under their own definition, which has its own note in the next section.
The question to ask about any benchmark
What denominator did the respondents use? If the publication does not say, the figure cannot be compared with yours at the precision it is quoted to.
Most publications do not say. A few do, and those are worth more than the rest combined.
Comparing against yourself
The reliable comparison. Same definition, same population, consecutive periods.
It gives up the ability to say whether the firm is good relative to others, and retains the ability to say whether it is changing, which is the question most decisions actually need.
The internal benchmark
Between teams in the same firm, under one definition applied by one person. Closer to a fair comparison than anything external.
Its weakness is mix: two teams with different seniority profiles and different commercial models are still not comparable, and the shared definition can disguise that.
When an external figure is genuinely needed
In a transaction. An acquirer will ask, and the answer has to be a single figure with a stated definition.
Here the right move is to state the definition explicitly and expect to be asked to restate on their basis, rather than to find the most flattering number available.
What to publish internally about benchmarks
If a benchmark is quoted in the reporting pack, the source and the survey's definition go with it. Where the definition is unknown, say so on the slide.
A benchmark with "definition not published" beside it is used carefully. One without that note becomes a target within two quarters.
What the quoted precision implies
A benchmark reported as 74.3 per cent implies the underlying figures were comparable to a tenth of a point. They were not: they were self-reported under unpublished definitions by a self-selected sample.
The precision is a presentational artefact and it does real damage, because a figure quoted to a decimal place is treated as a measurement rather than as an estimate with an unknown error.
Rounding a quoted benchmark to the nearest five points before using it is a crude correction that is closer to the truth than the original.
Using a benchmark to argue for something
The common use, and worth naming. A benchmark is quoted to support a target somebody already wanted.
In that mode the survey is evidence rather than information, and the definition question never gets asked because asking it would weaken the argument.
The defence is procedural: any benchmark entering a decision arrives with its source and its definition, or it does not arrive. That rule is easy to state, mildly awkward to enforce, and it removes most of the unreachable targets described in the next section before they are ever proposed.
The internal time series as the better benchmark
Eight quarters of the firm's own figures under one definition is a stronger comparator than any published average, because the definition is known and constant.
It answers the question a benchmark is usually being used to answer: are we getting better or worse.
What it cannot answer is whether the firm is good in absolute terms, and no benchmark with an unpublished definition can answer that either.
The practical rule
Use benchmarks to detect order-of-magnitude anomalies. Use your own history for everything else.
That split costs nothing, keeps the useful part of external data, and removes the part that produces unreachable targets.