[nodeval] some thoughts on evaluation planning stage

[Saul Albert]

Dear All,

Having read some of your comments over the last few days, and the
first 'planning' section of the evaluation guide pdf that was sent to the
list, I have a few thoughts that might be useful - Ruth, when you start
with the homework! :)

- There is no budget for the evaluation

  As with many parts of the project, there is no budget for this bit,
  unless I am mistaken. The guide mentions things like 'freelance artists
  should be paid for attending evaluation sessions'. Clearly that's not
  possible - but an evaluation has to take place. I'm not sure how it
  effects the process though. So far, we have traded volunteerism for
  freedom - you volunteer to do something, and you get control over the
  outcome/process. This approach has worked well for some things, and has
  contributed to the uniqueness of NODE.L's process, but it's worked badly
  for others. I reckon it would work badly for the evaluation process,
  unless we take it into account.

  I suggest that the way we take it into account is that we separate out
  the more quantitative processes, reflection and analysis that is
  necessary and of most interest to funders, from the more qualitative and
  engaging material (snowballs! snowballs!) that may be more widely
  engaging. Tim - you've already said that you have ultimate responsibility
  for reporting to ACE, so maybe this is already taken as read, but I think
  it would be useful to articulate the distinction between evaluations that
  form part of a contractual obligation, and evaluation for learning,
  self-indulgence and review. 

- Do we have to be as ambitious about the review as we have been about
  everything else?

  We've taken the 'burn it down, build it from scratch' approach to most
  things in NODE.London - we probably dont' want to do that with the
  evaluation process, but I think it's important, and one of our core
  objectives to think critically about the process and try to inform the
  notion of evaluation in arts practice itself through this process. I
  think there are aspects of NODE.London (the volunteerist strategy being
  one of them) that make this a probable outcome of this evaluation.

- We need SMART (specific, measurable, achievable, realistic, time-based)
  objectives

  This is good advice from that PDF - we have too many aims, not enough
  objectives, and the objectives are kind of wooly and difficult to
  measure. I'd propose cutting down to one or two main aims from the list
  of four, and objectivising the rest of them - which I think would work.
  Does this make sense?


- Timescales: we could make a distinction between aims and objectives in
  terms of whether they are underway during the season, immediately after
  the season, or if they're long term aims.

- We could generalise our aims a bit more - they are a bit close-to-the
  nose right now. Could we make the two or so aims we single out for
  attention more general - widely applicable, or using less specific
  language. The objectives could be the place we go for real specificity.

- Planning Checklist

  for those of you without time to read it, here's the 'planning phase'
  checklist from that document.. useful, and worth a critical read:

  - Who will have overall responsibility for the evaluation?
  - How much time and money is there for it?
  - Have review sessions been timetabled?
  - What is the main aim, and specific objectives of the project?
  - Is there consensus on that last point?
  - Are they realistic?
  - Are they time-based, and have they been thought through in different
    time-scales (short term/long term)
  - How do we get hold of unexpected results?

---------------------------------

I hope that's useful.

Oh, and Ruth, I do agree that maybe skype is not the best way to get
snowbally, but perhaps we could use it for something. I'm finding the
idea of a load of remotely collected interviews using skype and a
recording device very compelling for qualitative/publication purposes.

Cheers,

Saul.







-- 
-- http://chinabone.lth.bclub.org.uk/~saul/