Posts filed under ‘General’
Help wanted -5-point Likert or 10-point numerical?
Here’s one for you our readers.
Benchpoint is currently designing a survey for a client. Most of the questions have 5-point Likert scales:
Very satisfied
Satisfied
Neither satisfied nor dissatisfied
Slightly dissatisfied
Very dissatisfied
However the client wishes to have one question with a 10 point numerical scale where 9 is extremely satisfied and 0 is extremely dissatisfied.
We say we should stick to the same scale throughout the survey, and that a 5-point descriptive scale is better that a 10-point numerical scale.
What do our readers think?
Summer 2010 – Professional Development Workshop Series – evaluation and applied research
Claremont Graduate University (California, USA) has announced its Summer 2010 Professional Development Workshop Series on a variety of topics in evaluation and applied research. From August 20-23, you can participate in them directly in California or join them online for interactive webcasts:
Information and Registration>>
AEA365: Hot Tips, Cool Tricks, and Rad Resources for Evaluators
The American Evaluation Association have launched in 2010 a new blog AEA365 which aims to publish daily tips and resources for evaluation.
The blog is well worth a read – here is a post I contributed on evaluating communications and the “theory of change”.
New website for Benchpoint
![Benchpoint_thumbnail[1]](https://intelligentmeasurement.net/wp-content/uploads/2009/10/benchpoint_thumbnail11.jpg?w=455 "Benchpoint_thumbnail[1]")
The authors of this blog both work with Benchpoint – who have a new website – They have tools to make your measurement really intelligent.
(That’s enough plugging – Ed).
Richard
wanted: Survey data analysist
Here is an opportunity for those skilled in qualitative data analysis:
CONSULTANT – SURVEY DATA ANALYSIST
The International AIDS Society (IAS) is the world’s membership society for individuals working professionally in HIV, bringing together individuals and organizations from around the world to address current issues in HIV basic, clinical and prevention science. The IAS has over 13,000 members from 187 countries, and is based in Geneva, Switzerland.
Initially created to oversee the International AIDS Conference, the world’s premier gathering on HIV/AIDS, the IAS and its role in the global response have changed considerably since then, particularly in the past five years. This includes among others support to the professional development of key stakeholders coming from resource-limited countries, contribution to global and regional advocacy efforts, mobilization of regional partners through strategic regional partnerships.
In order to strengthen the evaluation of IAS activities, the IAS Evaluation Coordinator is seeking additional human resources who are qualified in survey data analysis.
Description:
Under the direct supervision of the IAS Evaluation Coordinator, the consultant will undertake the following tasks:
– Analyze responses to survey open-ended questions following IAS guidelines;
– Prepare summary reports.
The following skills and qualifications are essential:
-Excellent analytical skills;
-Excellent synthesis skills;
-Excellent reporting skills;
-Excellent written and oral English language communication skills;
-Strong computer skills in the Windows environment (particularly Excel and Word);
-Ability to work under pressure in an international context;
-At least 3 years of experience in monitoring and evaluation, of which at least one year in survey data analysis.
The following skills and qualifications are advantageous:
-Acquaintance with HIV/AIDS field;
-Excellent written and oral German and/or Russian language communication skills.
Type of contract: consultant (the duration will depend on the work scope but may vary from 5 to 20 days, with possible extension)
Location: home based
Candidates keen to take up this challenge and meeting the requirements should send their CV, motivation letter and availability period over the next 15 months to recruitment@iasociety.org by 15th September 2009.
World Metrology Day and Evaluation
As a blog dedicated to the art of intelligent measurement, we cannot let World Metrology Day go by without a fanfare.
According to the day’s organisers Metrologists are scientists who specialise in measurement techniques, so the team at “Intelligent Measurement” and Benchpoint can justifiably add a new descriptor to their various activities.
I can demonstrate how far intelligent measurement has come by describing my day’s activities. I am finalising charts and data for a big survey we are running for the Berlin Measurement Summit on how PR people and communicators measure their success. (There is still time to take part if you are in the business – www.benchpoint.com/measure.html).
I am also working on data to measure the effectiveness of the European Broadcasting Union’s recent conference in Lucerne, and putting together the finishing touches to a study measuring the effectiveness and reach of a UK professional association’s communications activities. For another client, a large international construction group, we are halfway through a measurement of its organisational effectiveness; for another we are putting together a programme of regular monthly surveys to take the management “pulse” in a fast changing environment.
These are all measurements which will put a substantive value to things which were only recently regarded as intangible, and immeasurable. Many organisational functions were judged by subjective discussion, and levered by political manoeuvring. Through intelligent measurement, it is now possible to allocate budgets and resources to projects and techniques which are the most likely to bear fruit in achieving organisational change or excellence.
Happy World Metrology Day!
Richard
US summer program on evaluation
If you are not able to go to Turin and if you are based in the US or close by, you might want to consider the summer evaluation courses taking place in Atlanta, Georgia, USA (pictured above) from 14-17 June 2009.
Organised by the American Evaluation Association and the Centers for Disease Control and Prevention, there are more than 40 workshops on offer over four days, in addition to keynote speakers. Here are some workshops that I would love to attend:
- What Counts as Credible Evidence in Contemporary Evaluation
- How Policy Is Made and How Evaluators Can Affect It
- Making Your Final Report Useful and Used
- Using Social Network Analysis in Program Evaluation
- Taking it Global: Tips for International Evaluation
- Every Picture Tells a Story: Flow Charts, LogFrames
- Evaluation 2.0: Measuring Connected Communications
11 hints for a successful evaluation
What makes a successful evaluation? I don’t mean a *success* in terms of producing *pleasing* results – but more so an evaluation that is successfully managed, respects all stakeholders and is of practical use to end users…I’ve been giving it some thought and I’ve come up with 11 hints for a successful evaluation, summarised in below and available with explanatory texts as a fact sheet (pdf):
- Make clear the unclear from the start
- Learn about what you are evaluating from the start
- Invest in planning
- Verify verify verify
- Communicate communicate communicate
- Collect only information you need
- Expect the unexpected
- Report your findings in a brief illustrative way
- Don’t hide the limits of your evaluation
- Don’t be disappointed if nothing happens immediately with your findings
- Give something back for all that you are taking
View the fact sheet with explanatory texts (pdf)>>
Do you have any other hints to add?
Eleven evaluation blogs
The American Evaluation Association has published a list of 11 blogs that focus on evaluation – we are happy to see that Intelligent Measurement blog is amongst them – along with many other interesting blogs. Certainly more evaluators could/should be blogging?
95 theses on evaluation
Disturbed by the state of affairs in evaluation, Professor Cronbach and colleagues wrote a 95 theses on reform in evaluation (inspired by Martin Luther’s 95 theses in 1517). They speak of the need for:
“A thoroughgoing transformation. Its priests and patrons have sought from from evaluation what it cannot, probably should not, give.”
Although written 28 years ago, the 95 theses (pdf) makes may pertinent points still valid today.
Here are several favourites that have stood the test of time (no. 75 is my favourite):
9. Commissioners of evaluations complain that the messages from evaluations are not useful, while evaluators complain that the messages are not used.
35. “Evaluate this program” is often a vague charge because a program or a system frequently has no clear boundaries.
49. Communication overload is a common fault; many an evaluation is reported with self-defeating thoroughness.
75. Though the information from an evaluation is typically not used at a foreseeable moment to make a foreseen choice, in many evaluations a deadline set at the start of the study dominates the effort.
95. Scientific quality is not the principle standard; an evaluation should aim to be comprehensible, correct and complete, and credible to partisans on all sides.
Read the full 95 theses (pdf) – despite this poor copy it’s well worth a read. The 95 theses originally appeared in the book “Towards reform of program evaluation“.
Glenn
intelligent measurement 