I see evaluation everywhere…

Isn’t it nice to read something well written and evenly paced about policies that can be fraught with well meaning, anecdotally based theory?

Reading this article brings to mind a few thoughts:
(1) be very cautious about correlation,
(2) those are some interesting studies described, how’d they do that?*
(3) what other assumptions are invalid, and
(4) more than ever, I’d like to see evaluations of policies and implementation tools be the norm, not the exception.

From Walter Frick for The Atlantic: A Strong Welfare State Produces More Entrepeneurs


* And really, how? Did they interview people as well? Were there multiple lines of evidence? Who paid for this research, because that’s also fascinating? Was it difficult, or did the days fly by doing it? What tools did they use? Are these reusable measures, or context-specific? So many other questions.

(post published from my phone, because that’s what I was reading after dinner when I found the article)

Leave a comment

Filed under Uncategorized


The internet really is infrastructure. It’s the highway, waterway really, of information. Good or bad, in colour or monochrome, ‘online’ is simply a tool. This article ‘Net Neutrality wins‘ by BoingBoing (my go-to for the online and the whacky) really sums up the recent argument and its results. Undoubtedly, we will see more on this idea. Hopefully, the ideal of net neutrality can be maintained.

background image sourced from morguefile.com

Leave a comment

Filed under Uncategorized

Free access to journal articles

Not everything, but a good long list of articles that otherwise live behind a pay wall.

Happy reading!

World Day of Social Justice 2015 | Explore Taylor & Francis Online

Leave a comment

Filed under Uncategorized

The journal Basic and Applied Social Psychology has BANNED …

… inferential statistics.

I remember the first time I was exposed to null values, p values, and the lot. I found them baffling and bizarre. When well presented, I could follow, but I had a heck of a time making heads or tails of them on my own. I began to wonder if I were dyslexic (I’m not) or if I’d accidentally taken some kind of hallucinogen (I hadn’t). Letters and weird bits of images of words swirled in my head, never seeming to settle. (Which was weird, I’m not going to lie).

Over time, I pieced it together. I think. Nonetheless, the early trauma remains and every time I do any kind of substantial statistics I must check, and recheck them, and then still ask for someone else to check them.

Now it seems that effort was all for not. Even confidence intervals are not safe (that 19 times out of twenty thing that everyone is used to hearing during media discussions of polls and surveys).

Mind, this is a journal mostly outside of my field, and not one I’m likely to which I’m likely to submit an article, so it’s impact on my work is not direct. But still. Who would have thunk it?

Maybe my teenage statistics self was the wiser self after all.


Leave a comment

Filed under Uncategorized

accountability is low hanging fruit, emphasis on low

One of the great things about having smart and insightful friends is that they share smart and insightful articles with you. Shout out to Stacy Ashton who found and shared this excellent blog on non profits. In the article Stacy shared, Vu Le from nonprofitwithballs makes a good case that viewed by the people being held to account, the concept is limiting and focused on punitive motivations. He argues that integrity and responsibility are more respectful and more successful motivators. Take a read and have a think about it.

Why we should rethink accountability as an organizational and societal value.

Leave a comment

Filed under Uncategorized

award nominees – harder than you’d think

Call for Nominations for 2015 CES Awards Extended Until February 27

I read this and thought, hey, what’s up with that? After a bit of investigating, it turns out that the CES hadn’t yet received nominees for any of its three awards. I had a quick think and came up with a list of names of people who would both qualify and be deserving of the awards. I contacted some colleagues, suggested the nominees and received nothing but enthusiasm. Well, right up until I contacted the potential nominees themselves. It seems that of the first three people I envisioned receiving awards, none actually were comfortable being nominated. They were each appreciative, but let me know they’d rather not be nominated. One wasn’t sure he’d attend, as his conference budget may be allocated to one of his staff member, each thought it was a bit too soon, and they all had a series of other of small reasons that essentially amounted to it didn’t feel quite right. I can respect that. I’m disappointed, not going to lie. Who knew this would be difficult? No wonder the deadlines were extended.

Leave a comment

Filed under Uncategorized

a thought occurred to me today

aea coffee break webinars
well worth the time

Today I attended an AEA Coffee Break webinar.1 Today’s session focused on evaluation tool kits. The tool kits discussed today were from a Farm to School program in Colorado. This was an interesting webinar in its own right. Without getting into the presentation at this moment (which was well done), a thought occurred to me while listening to some of the early description of the program.

There really are very different targeted beneficiaries whose interests align for this program when its successful. Students (children) who it is hoped will have improved nutrition, food knowledge, that kind of thing. Farmers, who want to see their products integrated into the local economy and are likely looking for some improved business results. The school district looking to keep its costs down and nutritional content up. (There may be others).

The question that popped into my mind right at the beginning of the session was, “Everyones’ interests align when the funding is secure, the context is consistent, and the program is headed toward a degree of success. What happens if things aren’t going well, if funding is cut (e.g. for budgetary or other reasons), or there is some other change in context (crop fail, teachers strike…)?”

Really, who is the more important beneficiary / stakeholder? Is there a favourite? What does the funder / program delivery organization do if the program goes badly? And, most tricky in my own mind, what happens if one stakeholder group’s needs must be preferenced over the needs of the others?

These are very much programming questions, it’s true. From an evaluation perspective though, how does the evaluation team tell that story? How do the evaluators capture the data to demonstrate whether or not an issue is in the offing; whether there is or is not a priority list; the consequences of a change in their context? How do they explain to someone that their needs will be secondary if things go sideways? Or more, interestingly, how does the evaluation team demonstrate the implications on various outcomes (which will necessarily vary by stakeholder group) what is likely to happen if the context or success of the program changes?2

It strikes me that if a program context changed or was likely to change, some evaluations wouldn’t necessarily catch the above questions – at least not if the program hadn’t thought about it in advance. (Am I wrong?) If no-one has worked through the implications of competing interests between different stakeholders at the front end, would it come up without a change in context.

Then, as I was typing this out and continued to see ‘context change’ as integral to explaining the issue that popped in my head, I got to thinking about John Mayne’s and others’ work on contribution analysis. That work is just full of capturing and describing context.

The thing is, I’m not sure my experience with contribution analysis is complete enough to capture the sceario I have in mind.3 When I think of contribution analysis, I think of an evaluation technique focussed on the front end of the initiative, the group of contributing factors needed to build solutions and in some ways on ascribing attribution to those working on and funding the initiative. I think of a scenario in which many groups are working together to support one, or a few similar beneficiaries.

When I’ve use contribution analysis before, the beneficiaries really weren’t that disparate. They were fairly similar really, with a collection of funders/programs working towards a similar goals. So, I haven’t used contribution analysis to look at context at the other end, when there are few funders/programs trying to simultaneously help different beneficiaries achieve some very different outcomes.

And that’s as far as my thoughts got before I finished coffee and needed to get back to other work.

If you have any thoughts, ideas, references, on this please let me know.

Also, it’s been a while since a drafted a long, rambling thought. Feel free to critique the style, tone, spelling, diction of this post.

All the best for now,

Today’s session was CFB 197: Evaluation Made Easy: Creating Meaningful Toolkits; a Farm to School example. Thanks to Dr. Lyn Kathlene from for taking the time to present today. I really appreciate your presentation style (aka clear) and a number of the implicit assumptions in your discussions (not everyone uses spheres of influence versus spheres of control, which I think are integral to evaluation. Similarly, I appreciate the ‘qualitative – quantitative data continuum’.)

Thanks also to AEA for coordinating and managing the series. If you haven’t taken advantage of these, I recommend them. The sessions are brief, so it is not too difficult to carve out the time from one’s work day to attend. There is a good variety of topics.

Understand, once I got thinking down these lines, I was no longer limiting my questions to just this example. I really have no idea how Dr Kathlene or her team would deal with that. The 20 minute presentation is no place to address it. Also, now I can think of all kinds of programs that might try to simultaneously support multiple beneficiaries.

3 I’ve been able to work with Steve Montague and Performance Management Network on a project or two.

Leave a comment

Filed under Uncategorized