The overall project aims to understand the nature of the DSS discipline using literature analysis. There have been a number of other critical reviews of DSS publications. Sean Eom’s series of analyses have used bibliometric approaches, including co-citation analysis, to analyse the intellectual structure of the field (Eom, 1995, 1996, 1999; Eom and Lee, 1990, 1993). Other reviews have examined the content of articles but have usually concentrated on only one aspect of the field. For example, Benbasat and Nault (1990) examined empirical research while Pervan (1998) analysed group support systems. The literature analysis at the heart of this project included all DSS types. It involved the protocol-based content analysis of each paper in the sample. This form of data capture has the disadvantage that it is a very labour intensive process but, importantly, it has the advantage that it can illuminate the deep structure of the field in a way that is difficult with citation studies.

The time period of published research chosen for analysis in this project is 1990 to 2004 (although some of the earlier papers that reported on parts of the project ended their analysis in 2002 or 2003). The start of this analysis period is marked by two much-cited reviews: Eom and Lee (1990) and Benbasat and Nault (1990). Both of these reviews covered the DSS field from its inception to the late 1980’s. A third review paper focusing on DSS implementation (Alavi and Joachimsthaler, 1992) provides a further anchor for the starting date of our analysis, as does the TIMS/ORSA and National Science Foundation sponsored discipline assessment (Stohr and Konsynski, 1992). The period 1990 to 2004 also marks an interesting period in the development of the information systems discipline because it witnessed a significant growth in the use of non-positivist research methods. Also, in industry, the analysis period saw the deployment of several new generations of DSS, especially the large-scale approaches of executive information systems (EIS), data warehousing (DW), and business intelligence (BI). To help identify trends in DSS research, the sample was divided into three five-year eras: 1990-1994, 1995-1999, and 2000-2004.

The sample of articles used in the project is shown in Table 1. We adopted a large set of quality journals as a basis of the sample because we believe that this best represents the invisible college of DSS research. Previous analyses of information systems research have used a similar sampling approach (Benbasat and Nault, 1990; Alavi and Carlson, 1992; Pervan, 1998). Alavi and Carlson (1992) used eight North American journals for their sample. However, Webster and Watson (2002) have criticised the over emphasis on North American journals in review papers. In response to this criticism, we included four European information systems journals (ISJ, EJIS, JIT, and JSIS) in our sample. The quality of journals was classified as ‘A’ level or ‘Other’. This classification was based on a number of publications that address journal ranking (Gillenson and Stutz, 1991; Hardgrave and Walstrom, 1997; Holsapple et al., 1994; Mylonopoulos and Theoharakis, 2001; Walstrom et al., 1995; Whitman et al., 1999) and on discussions with a number of journal editors. The articles were selected electronically by examining key words and titles. A manual check was performed of the table of contents of each issue of each journal. In addition, the text of each potential article for analysis was examined to verify its decision support content.

The sample comprised 1,093 papers that concern the development and use of IT-based systems that support management decision-making. Table 1 shows the distribution of these papers by journal as well as identifying the percentage of papers in each journal that were classified as DSS. Overall, 15.2% of published papers between 1990 and 2004 were in the DSS field. When only the general IS journals are examined, the proportion of DSS articles is still a healthy 11.4%. Each of these measures indicate that DSS is an important part of the IS discipline.

The protocol used to code each paper appears in Arnott and Pervan (2005). Some papers, termed ‘example articles’, were selected as being representative of the various article types. To calibrate the coding process, the example articles were coded independently and compared. A small number of changes to the initial assessments were made. The remaining articles were then coded by the two authors and a research assistant working independently. The time taken to code each article varied considerably, ranging from over an hour for large, complex papers, to ten minutes for the straightforward coding of a known paper. In coding each paper the emphasis was on the dominant attribute of each factor for each paper. For consistency, the coding of articles by the research assistant was reviewed by the first author. The coded protocols were entered into an SPSS database for analysis by the second author, who also performed statistical consistency checks on the coding.