Economic Evaluation: Objective, approach, challenges and solutions

Keep them Safe (KTS) is the NSW whole-of-government response to the recommendations of the Wood Special Commission. NSW has spent nearly $800m over five years to reform the child protection and early intervention systems, with over 50 different projects being funded as well as changes to the legal system and in protocols for interagency collaboration. The KTS evaluation has been the most ambitious social services evaluation undertaken by the NSW government in recent history. The success of an evaluation of this scale depends on successes in reaching across disciplinary, organisational, and sectoral boundaries. Its completion represents an opportunity to reflect on such large scale, complex evaluation projects and consider how they can be undertaken going forward.

The NSW Department of Premier and Cabinet commissioned a research consortium led by the Social Policy Research Centre, UNSW Australia, to evaluate the outcomes of KTS, focusing on outcomes related to system change, improvement of services and the impact of KTS on vulnerable children and families in NSW.

This panel will discuss the complexities and challenges of designing, procuring, managing and conducting the KTS evaluation; from the perspectives of the commissioning agent and researchers. A key innovation of the evaluation was the construction of a unique and extensive area level child wellbeing database across the 153 NSW Local Government Areas (LGAs). The database included numerous key outcome measures, government investment and a range of local demographic, social and economic indicators. The evaluation team also built an individual-level longitudinal database using de-identified administrative unit-record data spanning three major internal databases. The panel will examine the challenges and potential of constructing such databases, with particular reference to administrative government data holdings. The panel will also discuss lessons learnt and recommendations for future evaluations of multi-agency, large scale policy reforms.