In this section, study logistics are described. The overall study design is reflected in Figure 1.2. Sub-study methods and methodology are addressed in the individual chapters of the volume. Some details of the case-study method are included below in order to avoid repetition across the seven state case reports.
The study process was a combination of deductive, top-down broad definitions of aims, questions and procedures, and inductive, bottom-up consideration of sources of evidence, project resources and feasibility. It could be said that early results were mainly inductive, these being followed by a more deductive, top-down review.
Individual state teams managed their respective case studies, with the overall project managed by Guy Gable (project leader) and Bob Smyth (project manager). The main mechanisms of project management were: 1) a project web site; 2) regular status meetings of the project leader and project manager and related project status reporting by e-mail; 3) face-to-face team meetings when feasible (on several occasions in conjunction with ACIS or the Australian Council of Professors and Heads of Information Systems [ACPHIS]); 4) several teleconference and Skype sessions, as a cost-effective alternative to face-to-face meetings; 5) the case-study protocol served as a valuable mechanism for coordinating the study teams.
A range of complexities was encountered throughout the study, some of which were fully or partially overcome; others of which were not. Several pertained to the scope and object of the study: academia versus the profession versus both; teaching versus research; past versus present. There was discussion about the utility of ‘state’-based case studies, it being mooted that other groupings of institutions might be more meaningful: for example, ‘Sandstone’ universities versus the others; large IS versus small IS—that is, based on numbers of undergraduate IS students).
At a meeting convened by Gable in Canberra shortly before publication of the December 2006 special edition of the AJIS, the possibility of extending the study was raised. The meeting agreed that a book would allow the results of the research to be available in a more permanent form—a desirable situation given the potential value of the studies to the IS discipline and given the plan to replicate the study over time. It was agreed that researchers who had authored papers in the AJIS special edition be invited to participate in this second phase.
Subsequently, 17 of the 18 researchers who had contributed to the AJIS special edition agreed to contribute to the second phase; only the second South Australian study, conducted by Metcalfe, was not taken to phase two. An additional contributor, however—Wilson—agreed to prepare an alternative view of IS in New South Wales, as a counterpoint to the earlier findings published in AJIS. As described earlier in this chapter, the extensions in phase two involved a new analysis of the state data in relation to the extended Ridley framework; they also provided for expanding on earlier material and focusing on a sample of important contributors to the development of the IS discipline, via selected vignettes.
The case studies were largely exploratory and descriptive, with relatively less emphasis on interpretation and the ability to be generalised. The case-study approach is well suited for investigation where there is little established theory on the topic (Yin 2003). The case-study method also has the advantage of allowing the researcher to develop a good feeling for the complexities of interacting forces and subtleties that are more difficult to detect with other methods. Walsham (1995) refers to the ‘rich insight’ possible from the case-study approach.
The data collected in the case studies focused on questions implicit in the case-study protocol, as described earlier. It was planned that team members in Australian states and territories (referred to collectively as the study ‘states’) would gather qualitative data about each university (as well as relevant state-level data). The case-study method was agreed to as the research approach for the team study. The data gathered were intended to provide insights into the distinctive characteristics of IS in each university in relation to: the degree of administrative autonomy, size and influence, curriculum, research, local influences and significant people shaping IS at that university. Anchoring the data gathering and analysis in phase one was Gail Ridley’s (2006a) evolving draft framework derived from theory on the emergence of disciplines. Team members in each state sought to analyse the data across the universities engaged in IS in that state, and to present general observations on the features of IS as an academic discipline in that state. Phase two provided further analysis of this state data, taking account of the updated version of Ridley’s framework described in Chapter 3 of this volume. Phase two also entailed a consolidated analysis of the data from the study.
Yin (2003) argues for the use of a case-study protocol to guide any study employing the case-study method. To this end, study team members developed a multi-state case-study protocol. An abridged version of the protocol is included in Appendix 1.1. A detailed version of the protocol was reported in the AJIS (Smyth and Gable 2006). It was intended that use of the protocol would contribute to:
comparability across the states
consistency across the individual case studies
efficiency in the conduct of the case studies, with potential for data gathering and some analysis being delegated to research assistants or other junior researchers.
Yin (2003) strongly favours building a protocol around relevant theory. In this study, the protocol relies on a framework, the emergence of a discipline, developed by Ridley and articulated in preliminary form in the 2006 special edition of AJIS, and in expanded form in Chapter 3 of this volume.
Also, many of the tenets of general systems theory (Ackoff 1971) are implicit in the approach to data gathering and the themes and questions documented in the multi-state case-study protocol. The approach to data gathering advocated—based on semi-structured interviews utilising broad themes to tap the perceptions of interviewees—is consistent with general systems theory. This approach permits the researcher to take a more holistic approach to the topic, and allows the interviewee to touch on the multiplicity of interacting factors that might contribute to the distinctive characteristics of the IS presence in each university. The protocol directs the researcher to just some of the potential interacting factors that might determine the distinctive characteristics of IS at a given university: that the interaction of geography, the administrative structure and individuals from within and outside the university could, over time, influence curriculum and research at that university. This planned concern for pursuing a holistic view by calculated consideration of factors beyond those suggested by the guiding framework proved useful in the subsequent consolidated analysis of state data.
Given the descriptive and exploratory character of the overall study, the team harboured no illusions regarding the ultimate completeness of issues to be identified, related evidence to be gathered and analyses to be conducted. It was acknowledged that the study offered a mere starting point for continuing monitoring of the state of IS in Australia. Regardless, efforts were made to achieve some level of representativeness of the evidence and perspectives reported. Key mechanisms were: 1) selection of the study team; 2) review of draft state reports by interviewees; 3) review of state reports by intrastate experts; and 4) global review of the draft special issue by appropriate experts.
In establishing the study team, Australia-wide representation was sought; this suggested state-based case reports. Senior and long-standing IS academics were approached; in most cases, those contacted first welcomed involvement but with some changes to the composition of the team over the course of the study.
All interviewees received an early draft of the state report in which their views were recounted. Feedback was channelled through the project manager back to the state teams, and the state teams implemented changes.
In addition to careful review by state team members, interviewees, the project leader and the project manager, each state report was given further local exposure in draft form before wider circulation to cross-case reviewers. Selected ‘local experts’ were sent a copy of the draft report for review, the aims being to:
minimise potential adverse reaction from perceived misrepresentation
try to ensure the report was as representative of the state as possible
enrich the report with further insights
ensure that the process of peer review resulted in chapters of a strong academic standard.
Two senior and internationally notable IS academics, Iris Vessey and Bernard Glasson, were sent a draft of the complete phase-one output—as prepared for the AJIS 2006 special issue—and were requested to review the material and respond to a brief set of questions pertaining to the historical evolution of IS-in-Oz, the current state and the possible future of IS-in-Oz. The aims of this survey included: 1) gathering further insights; 2) accounting for diverse perspectives; 3) keeping the study team accurate and seeking balance (soliciting third-party reactions to our areas of emphasis and our interpretations); and 4) maximising rigour.
Though extensive measures have been pursued to ensure representative input and a balanced report, resource and time limitations have constrained what is possible. While such a report will unavoidably reflect certain emphases and biases and choices made at all stages of its production, the team nonetheless apologises for any omissions or oversights. Given the desire that this study be replicated in future, feedback on any such omissions or oversights is all the more welcome.
The overall study effort was compounded substantially by the intention to document the approach for repeatability. Alvesson and Sköldberg (1999) use the term ‘reflexive methodology’ to refer to an evolutionary approach that aims to maximise the quality of study results. We prefer the term ‘methodological action research’ (MAR) whereby, in addition to results in relation to research questions posed, the study seeks generalisable contributions to knowledge as regards the research process. We define MAR as:
an action research approach to studying the process of research; it is a reflexive process of progressive problem solving led by individual researchers, possibly working with others in a team, to improve the way they address research issues and solve research problems. MAR is conducted above and behind the main research activity, with the researcher, on this second level, observing themselves (and their team) and their experience of the research process, the intent being to better understand and improve that process and to document related methodological learnings.
We therefore sought to establish a highly systematised and proceduralised approach, readily extended and repeatable across other countries and regions (and publication collections—for example, conferences or journals; see ACIS Archival Analysis in Figure 1.2). Note that there is a dotted arrow pointing up from the MAR oval in Figure 1.2 to a similar study at the IS-in-PA level of that figure. This reflects how MAR learning on the IS-in-Oz study informed the parent study by evolving, recording and piloting various methods, mechanisms, concepts and interpretations. It is noted that several of the IS-in-Oz research instruments and approaches needed only marginal adaptation for the purposes of the IS-in-PA study. It is noted also that much of the MAR learning from the IS-in-Oz study was reported in the ‘Methodological learnings’ article in the CAIS special issue (Gable and Smyth 2007).