An effective and economical evaluation is a vital part of
achieving, and demonstrating the achievement of, value for money in
a government approach to the market.
Done well, the process will document, on the face of the record,
compliance with the legislative and procurement policy framework
and legal process and probity requirements.
Our recent experience in numerous evaluations has revealed the
following common mistakes in evaluation:
Not having a proper methodology for treating risk, particularly
double counting risk
Double counting risk will provide an inaccurate summation of the
whole evaluation, as emphasis will be placed unnecessarily on a
given risk category. risk should be assessed in the areas of
technical Worth, price and legal (relating to compliance with the
deed of standing offer and issues for negotiation).
A risk matrix detailing ?Measures of likelihood? and ?Measures
of consequences/impact? following usual risk management principles
can be used.
Our experience indicates that it is important to comply with the
iso310000 risk standard, however anZs 4360 is still very useful and
practical. Accordingly, a risk description matrix following the
anZs 4360 standard, guidelines and handbook modified for the needs
of the project and covering Measures of likelihood and Measures of
consequence/impact are generally very useful.
Including a risk description in your rating scale for
a response with high technical merit may not always be low risk.
the two issues are separate and generally need to be treated
scoring by numbers rather than by descriptions.
As indicated above, scoring by numbers is often not helpful and
fails to describe properly why a particular rating has been
determined. Assessing and rating primarily by reference to numbers
rather than word pictures opens an evaluation to criticism or
Inadequate word pictures - not lining up with the rating scale.
There is always subjectivity in evaluation. Word pictures are
critical in demonstrating an absence of bias and form the
evidential basis for judgments made. Reliance on numbers can become
subjective and any scoring should be adequately described and
Not structuring process and templates so assessments,
evaluation reports and debriefings can be easily done from the same
Utilising the same data source in an assessment is essential
given it allows the user to refer back to an agreed base in order
to measure appropriateness of evaluation. This will avoid any
inaccuracies on measurements and ensure effectiveness of the
probity and evaluation process.
Poor use of evaluation templates and/or evaluation software,
including not properly adapting them to your process/evaluation.
Understanding the software being utilised is essential and will
dictate the efficiency and suitability of the evaluation plan. Very
often software beyond simple excel spreadsheets (properly checked
and verified) does not provide the structure or flexibility
required for many evaluation processes.
Inappropriate rating scales (too many points on the scale or
Utilising the correct amount of categories is important and this
should be adapted for each evaluation having regard to the nature
of the requirement and the risk involved.
Not differentiating between panels and first past the post
Not following the evaluation plan or process and failing to
manage and record steps and outcomes properly.
Illegitimately double counting good points or bad points.
As noted above, double counting risk will provide an inaccurate
summation of the whole evaluation given emphasis will be placed
unnecessarily on a given risk category.
Our experience in designing and implementing evaluation
processes highlights the value of having systematically
decipherable methods and documentation which includes appropriate
flexibility. This can significantly ease work load and increase
consistency, efficiency and confidence in the evaluation.
The content of this article is intended to provide a general
guide to the subject matter. Specialist advice should be sought
about your specific circumstances.
To print this article, all you need is to be registered on Mondaq.com.
Click to Login as an existing user or Register so you can print this article.
There has been a range of recent legal developments that affect privacy, child abuse claims and workers compensation.
Some comments from our readers… “The articles are extremely timely and highly applicable” “I often find critical information not available elsewhere” “As in-house counsel, Mondaq’s service is of great value”
Register for Access and our Free Biweekly Alert for
This service is completely free. Access 250,000 archived articles from 100+ countries and get a personalised email twice a week covering developments (and yes, our lawyers like to think you’ve read our Disclaimer).