Evaluation of Investments in the Strengthening Management and Governance Programme
Table of contents
- Executive Summary
- 1. Evaluation of Investments in Strengthening Management and Governance
- 2. What Has the SMG Programme Achieved?
- 3. Possible Enhancements to the SMG Programme
- 4. Conclusion
- Appendix One: Methodology
- Appendix Two: Results of the SMG Online Survey
- Appendix Three: References
3. Possible Enhancements to the SMG Programme
While the SMG programme was well received by organisations and assessors, the evaluation identified areas of possible improvement.
At a programme level, the evaluation identified enhancements to refocus the programme and provide more targeted assistance to meet the changing needs of organisations. These enhancements include:
- broadening the qualifying criteria for organisations, including the funding threshold; • offering SMG as a total development package;
- refocusing SMG with a targeted and tailored approach; • offering mentoring opportunities;
- assisting organisations with information sharing;
- improving the preparation of assessors before the assessment; and
- matching assessors more closely with organisations.
At a policy level, the evaluation identified some strategic issues that may have future implications for the SMG programme. The nature of these issues focuses on the development needs of organisations as follows:
- Formalising training programmes.
- Providing access to governance resources.
- Increasing public knowledge of Māori governance.
- Redefining the SMG programme outcomes.
- Brokering support and collaboration.
3.2 Refocusing the SMG Programme
Broadening the qualifying criteria
To qualify for the SMG programme, Māori organisations must have government contracts in excess of $300,000 per annum. While this benchmark may be useful in managing expectations around access to the SMG programme and in targeting more prominent organisations with accountability for government contracts, assessors believe there is a large number of organisations that need governance and management support but fail to qualify under the existing criteria.
The SMG programme could be useful to more organisations if the qualifying criteria were broadened to allow non-government-funded organisations to qualify and if the contract threshold was lowered (or removed) to allow low income generating Māori organisations to participate in the programme. Any changes to the criteria should be balanced against the original intent of the SMG programme in order to maintain the integrity of the programme.
SMG as a total development package
Organisations considered that the SMG programme could be improved by tailoring the programme as a complete package covering initial assessment, and remediation work, followed by a post-assessment monitoring and evaluation phase. Organisations commented that the current SMG programme was more effective when delivered in a structured package (i.e. with assessment and remediation).
Currently, some Te Puni Kōkiri regions actively promote the remedial phase, with other regions taking a less pro-active approach. Some organisations that participated in the assessment phase declined to carry on with the remediation phase, stating that they would undertake the work themselves, in line with their own resources and timeframes.
Some organisations were interested in having follow-up with their original assessors to share progress. In most cases it appeared there was no contact between assessors and organisations after the assessment phase. Some assessors also mentioned that they would be interested in checking in with organisations to see how their recommendations had been received and how the organisations had progressed in the intervening period.
A post-assessment monitoring and evaluation phase would be a valuable add-on to the programme, as it would enable organisations and Te Puni Kōkiri to track progress, ensure recommendations were implemented, and identify any new issues. It would also enable Te Puni Kōkiri to assess whether any improvements had been sustained over time.
Refocusing SMG with a targeted and tailored approach
One of the strengths of the SMG programme has been the standardised assessment process delivered by experienced assessors. The assessment framework and process have largely been a ‘one size fits all’ approach with some flexibility to tailor the approach to meet the specific needs of organisations. Some organisations suggested that, in addition to the standard SMG process, the programme could be enhanced by inclusion of a component that tailored specific advice and expertise to the explicit needs of the organisation, whether that be related to new business development, investment opportunities or sector-specific knowledge.
Some organisations expressed interest in receiving follow-up assistance in the form of coaching/ mentoring activities from experienced consultants. Rather than have consultants undertake all of the remedial work, organisations requested that they work alongside the consultants, where possible, to learn and retain the institutional knowledge and experience.
Some organisations would like access to the consultants over a longer period and for them to be available on an on-call basis to support and guide the organisation.
“It would be useful to have a mentor for six months or so who could attend meetings and help out with particular issues.” (SMG organisation)
Organisations considered there was value in being able to link up and share information with other similar organisations. While this already occurred on an ad hoc basis, largely with other organisations within the same sector, organisations expressed interest in establishing a more formal process. Such an initiative may be outside the parameters of the SMG programme; however, the evaluators included this issue as it was raised a number of times by organisations.
Assessors also commented that there could be value in assessors meeting to feed back on the SMG process and to share insights and experience to continuously improve the SMG programme.
Preparedness of assessors
Organisations commented that some assessors need to spend more time prior to the organisation visit to fully understand the organisation and the sector it operates in, so that during the visit less time is spent briefing the assessor on the background of the organisation.
In addition, the restriction on organisations working with their Phase 1 assessors during the remedial phase (Phase 2) meant that more time had to be spent bringing new consultants up to speed on the particular needs of the organisation. Time could have been reduced if the same assessors were used throughout both stages. However, organisations accepted the rationale behind this approach (i.e. independence and preventing self-generated work) although some still expressed frustration at what they regarded as lack of flexibility by Te Puni Kōkiri.
Matching of assessors with organisations
Most organisations accepted that they could not select their assessor (during Phase 1) and that there was value in the assessor coming into an organisation with a ‘clean slate’.
One organisation considered that the SMG programme could be enhanced if Te Puni Kōkiri were to match organisations with assessors who had background and knowledge related to the specific sector they were working in.
For example, one organisation was unhappy with what they perceived as the youth and inexperience of their assessors. They considered the assessors to be lacking an in-depth understanding of what is entailed in operating as a kaupapa Māori provider. Because of this perceived shortcoming, the organisation considered that the assessors’ questioning of board members as to their understanding of their roles and responsibilities failed to acknowledge the value placed by the organisation on the importance of cultural support and tikanga advice rather than on generic trustee skills.
3.3 Understanding the Development Needs of Māori Organisations
Formalising development programmes
One of the key themes the evaluation identified was the urgent need for governance training in Māori organisations. While governance training was high on the agenda for most organisations, finding relevant training or development opportunities specifically tailored to organisations such as marae or land trusts was often difficult. Formalising a training programme with the attainment of a qualification for trustees of Māori organisations could be an area of future work for Te Puni Kōkiri. This could be undertaken with the New Zealand Qualifications Authority and other governance training providers such as New Zealand Trade and Enterprise and the Institute of Directors in New Zealand.
The Hui Taumata Action Taskforce10 recommended the establishment of a Māori Leadership in Governance Development Centre for the delivery of a broad-based leadership and governance education programme targeted at a wide range of Māori organisations. This suggestion may provide a platform for the development and delivery of the requisite training programmes.
Access to governance resources
Survey results indicated that organisations wanted more governance resources to assist their development, including governance toolkits and templates.
Te Puni Kōkiri has developed some resources that could assist SMG organisations in their governance and management functions. The Boardwalk Series on governance, the governance website11 and the recently published Governance guide,12 a self-help guide for people involved in the governance and management of Māori organisations, are freely available to organisations.
Te Puni Kōkiri should ensure all organisations are aware of, and able to access, these resources as well as resources available from other agencies.
Increasing knowledge of Māori governance
A vast body of knowledge on Māori governance exists within the SMG programme and particularly with the assessors. No formal process has been undertaken to extract and utilise this information to inform policy development or to understand the development needs of Māori governance. Although this evaluation goes some way towards identifying issues in Māori organisations, there still remains much information that could be analysed, particularly in exploring what defines Māori governance and whether there are unique characteristics, and how active Māori organisations are in using Western practices of governance.
One assessor suggested Te Puni Kōkiri may wish to consider promoting the guiding principles of the Policy Governance Model13 as a basis for further development of knowledge about Māori governance.
Redefining the SMG programme outcomes
The ambiguity of the SMG programme outcomes made it difficult for the evaluation to measure whether these high level outcomes were achieved. For example, how does Te Puni Kōkiri define ‘strong Māori organisation’? Is it possible or desirable for Te Puni Kōkiri to attempt to determine if government funds administered by other agencies are targeted appropriately?
Te Puni Kōkiri should redefine the programme outcomes to ensure they are clear, measurable and meaningful and align with the Māori Potential outcomes. Te Puni Kōkiri should also consider further developing the outcomes model (outlined in section 1.4.2) and testing the relevance of the low order outcomes of the model, including, where possible, developing indicators and measures of success for these outcomes.
Brokerage and collaboration role
One of the early policy intentions for the SMG programme was for Te Puni Kōkiri to broker and facilitate access to resources from other agencies (instead of Te Puni Kōkiri funding both the assessment and remedial phases). To a large extent this only occurred on an ad hoc basis, at a localised level, and is dependent on the relationship Te Puni Kōkiri staff have with other agencies. The impetus for brokering agency collaboration for organisation development opportunities may have diminished somewhat now that the SMG programme is well established and organisations know that Te Puni Kōkiri can fund both phases. However, opportunities still exist for Te Puni Kōkiri to increase synergies with other agencies that offer provider development funding or have a vested interest in building the capability and capacity of Māori organisations.
10 Hui Taumata Action Taskforce and Victoria University (n.d.).
12 Te Puni Kōkiri (2007).
13 Carver (1997).