One of the most disappointing aspects of the NT intervention is that there was virtually no lead time to prepare or think about an evaluation framework. Some ex ante planning with Indigenous leaders and relevant bureaucracies would have both lessened the public resistance and facilitated evaluation. It will now be very difficult to evaluate the outcomes of the intervention because no groundwork was laid to establish credible benchmarks for what existed before the policy shift.

In the United States, most major change in policy would start with a strategy for evaluating the social experiment with random assignment (Bloom 2005). Social experiments are regarded with particular suspicion in Indigenous policy because they raise ethical questions about the treatment of the Indigenous population as subjects, rather than as people. The Howard government has clearly indicated a willingness to engage in paternalism with this intervention, but its failure to establish benchmarks or construct control groups means that it will be almost impossible to attribute any changes in outcomes to the intervention. For example, the rate of child abuse and neglect in the NT may go up or down depending upon the level of measurement error both before and after the intervention. Consequently, the NT intervention is unlikely to be held to account and the government can make almost any claim it wants about what happens as a result of its policy.

Given the scale of the intervention, it would not be unreasonable to characterise it as one of the largest social experiments in Australian history — with NT Indigenous communities being the ‘treatment group’. Of course, the ‘experiment’ analogy breaks down when one remembers that there has been no attempt to randomly assign people into control and treatment groups. Another problem for the analogy is that some of the policy initiatives are not entirely confined to the NT. Nevertheless, it is time to consider explicitly adopting a more scientific approach to experiments in Indigenous affairs — especially since it is, arguably, already occurring.

One potentially important advance for the evidence on the development processes facing Indigenous children is the Longitudinal Study of Indigenous Children (LSIC). The 2007 federal budget announced that LSIC would involve around 1,500 babies and children from 11 regional sites. The relatively small sample size may circumscribe the power of the analysis to discern between competing hypotheses. Another concern is that LSIC is not national in scope and hence can be accused of reflecting regional idiosyncrasies. One potentially unfortunate side-effect of the NT intervention is that it may disproportionately affect the level of cooperation of LSIC respondents there vis-à-vis other states and territories (that is, it might induce some measure of non-sampling error).

When LSIC data is eventually collected, it should enhance the findings from the cross-sectional studies of the Western Australian Aboriginal Child Health Survey (WAACHS, see Zubrick et al. 2006). While the proposed LSIC will not be a national study, it shares this limitation with the WAACHS.