What email address or phone number would you like to use to sign in to Docs.com?
If you already have an account that you use with Office or other Microsoft services, enter it here.
Or sign in with:
Signing in allows you to download and like content, and it provides the authors analytical data about your interactions with their content.
Embed code for: 1-s2.0-S0148296309000654-main
Select a size
Designing the spreadsheet-based decision support systems course: An application of Bloom's taxonomy Craig K. Tyran Department of Decision Sciences, College of Business and Economics, Western Washington University, Bellingham, WA 98225-9077, USA abstracta r t i c l e i n f o Article history: Received 1 December 2008 Received in revised form 1 February 2009 Accepted 1 March 2009 Keywords: Spreadsheet software Decision support systems Bloom's taxonomy Management education Decision support systems (DSS) have played an important role in organizations for many years. As DSS continue to be developed for industry applications, a number of business programs in universities offer a specialized course aimed at helping students better understand and develop DSS systems to support decision making. Spreadsheet software coupled with an application programming language can serve as a useful DSS generation software package for such courses. To help students develop their technical skills for spreadsheet- based DSS, the principles underlying Bloom's taxonomy of educational objectives can serve as a guiding framework for instructional design. This article describes how Bloom's taxonomy has been used to support the design of two different DSS courses, an undergraduate course and an MBA course. Student survey data collected over the past four years from students enrolled in nine sections of the DSS courses are reported. Based on the survey ﬁndings, as well as the instructor's observations, it appears that a teaching strategy based on Bloom's taxonomy offers a worthwhile framework for instructors who teach a spreadsheet-based DSS course. © 2009 Elsevier Inc. All rights reserved. 1. Introduction Decision support systems play an important role in organizations (Keen and Scott Morton, 1978). A decision support system (DSS) is a system that enhances the decision making process. A DSS typically offers the decision maker one or more computer-based tools that may be used to structure a decision making situation and enhance the quality of decision making (Marakas, 2003). As decision making is ubiquitous in organizations, numerous examples of DSS exist in the ﬁeld, ranging from simple spreadsheet systems to very large systems involving complex components such as simulation models and optimization models (Power, 2002). To help educate and train future business managers and software developers about DSS, a number of university business programs offer one or more courses that focus on DSS. Such courses have been implemented for students pursuing specialized areas of study such as management information systems (MIS) (Palocsay and Markham, 2002), as well as students in more general business programs such as a MBA program (Zobel et al., 2000). Published discussions of DSS curriculum issues (Ragsdale et al., 2002; Wehrs, 2000) note that a particularly important concern for DSS education relates to the question of how educators may help students to link the conceptual aspects of DSS, such as DSS components or modeling, to the practice of actually developing a DSS. DSS educators found students had difﬁculties bridging the gap from DSS theory to practice. Finding suitable DSS generation software that students could use to apply the concepts learned in the classroom was difﬁcult. While DSS generation software existed in industry, such software was not always useful for educational purposes, as the software could be difﬁcult to learn and could be expensive to purchase and support. In recent years, however, a useful DSS generation software platform for educational and industrial use has emerged: Microsoft's Excel spreadsheet software and the Visual Basic for Applications (VBA) programming language. By leveraging the Excel spreadsheet software with the VBA programming language it is possible to create powerful industrial grade DSS applications that include professional graphical user interfaces, sophisticated models, and connections to databases. For example, Fig. 1 shows an Excel- based DSS that includes an easy-to-use interface and links a spreadsheet to a database using VBA. Several authors have provided examples that demonstrate that the combination of Excel spreadsheet software and VBA can serve as a useful DSS generation software package for the university classroom (e.g., Palocsay and Markham, 2002; Ragsdale et al., 2002). Building upon the work of other educators, Excel spreadsheet software and VBA have been adopted as the DSS generation software for the undergraduate and graduate DSS courses taught at Western Washington University's College of Business and Economics. To provide a guiding framework for teaching spreadsheet software and related programming skills, the course designs build from the principles underlying Bloom's taxonomy of educational objectives (Bloom et al., 1956). Bloom's taxonomy categorizes cognitive skills Journal of Business Research 63 (2010) 207–216 E-mail address: firstname.lastname@example.org. 0148-2963/$ – see front matter © 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.jbusres.2009.03.009 Contents lists available at ScienceDirect Journal of Business Research into six hierarchical levels. Instructors' familiarity with Bloom's taxonomy is useful because Bloom's cognitive learning model can help instructors develop a teaching strategy that will support learning such that students can progress smoothly from lower to higher level cognitive skills (Lovell-Troy, 1989). Although research articles discuss the application of Bloom's taxonomy for many educational areas (Anderson and Sosniak, 1994), the literature does not discuss speciﬁc guidance and examples regarding ways to apply Bloom's taxonomy to a spreadsheet-based DSS course. A primarymotivation for this manuscript is toaddress this gap by discussing how Bloom's taxonomy has been used to support the design of undergraduate and graduate spreadsheet-based DSS courses. A second motivation for this manuscript is to extend the existing literature on spreadsheet-based DSS education by reporting on student survey data collected over the past four years. The study collected these data from students enrolled in nine sections of DSS courses, ﬁve sections of an undergraduate course and four sections of a graduate MBA course. Based on the survey ﬁndings, as well as the instructor's observations, it appears that a teaching strategy based on Bloom's taxonomy offers a worthwhile framework for instructors who teach a spreadsheet-based DSS course. The remainder of the article contains four major sections. The next section provides a review of the relevant literature, including an overview of Bloom's taxonomy. The third section describes how we used Bloom's taxonomy to guide the design of our undergraduate and graduate spreadsheet-based DSS courses. The fourth section provides an assessment of the learning approach by summarizing the ﬁndings from a student survey. The ﬁnal section of the article offers a discussion of the survey results and suggestions for future research work. 2. Literature review: Bloom's taxonomy Bloom's cognitive taxonomy was initially created as part of a project to provide a theoretical framework to facilitate communica- tion among educators regarding learning assessment and testing materials (Bloom et al., 1956). Those interested in academic assess- ment decided through a series of discussions that a classiﬁcation of educational objectives would provide the basis for curriculum development and testing, since educational objectives provide the basis for curriculum development and testing. Ultimately, theorists developed three different taxonomies of learning objectives to represent three domains of learning: a cognitive taxonomy focused on intellectual learning, an affective taxonomy concerned with the learning of values and attitudes, and a psychomotor taxonomy that addressed motorskillslearning.One cognitivetaxonomy(Bloometal., 1956) is known widely as the Bloom's taxonomy. This taxonomy recognized six levels of cognitive skill ranging from the lowest level skill of knowledge to the highest level skill of evaluation. Table 1 provides a summaryof the sixcognitive levels, along with examples of cognitive activities that are related to spreadsheet-based education and VBA computer programming. In addition to its value for the assessment of learning, instructors use Bloom's taxonomy to support the design of learning strategies (Anderson and Sosniak, 1994). At the higher education level, the taxonomy has been used to guide instructional design spanning a diverse range of subject areas, including business (Ainsworth, 1994; Athanassiou, et al., 2003), physical science (Pungente and Badger, 2003), social science (Lovell-Troy, 1989), and the arts (Hamblen, 1984). A general assumption underlying the classiﬁcations of the cognitive taxonomy is that a learner needs to develop mastery of a Fig. 1. Example of spreadsheet-based DSS using VBA. 208 C.K. Tyran / Journal of Business Research 63 (2010) 207–216 skill at a lower level as a prerequisite to moving on to related skills at a higher level (Krathwohl, 2002). For instance, consider the following simple example relating to spreadsheet-based DSS education. Sup- pose that an instructor is attempting to teach a student how to use a sophisticated type of spreadsheet formula like the LOOKUP formula in Excel. The taxonomy would suggest that a student needs to under- stand what the spreadsheet formula does and how it works at the comprehension skill level before being asked to perform an applica- tion level skill such as the insertion of the formula intothe appropriate spreadsheet cell. The taxonomy can help instructors to appreciate the wide range of cognitive skill levels associated with learning. By taking the different cognitive levels into account when designing a course, an instructor can develop a teaching plan that will consider a student's smooth progression from the lower to higher cognitive levels. A familiarity with the scope of cognitive levels may also encourage instructors to consider ways to develop a more ambitious teaching approach that will help students attain higher levels of cognitive learning (Chyung and Stepich, 2003). For example, in some cases spreadsheet education adopts a literacy focus that emphasizes the lower cognitive levels of spreadsheet skills (Davis et al.,1999). After a review of Table 1, instructors of such courses may recognize that higher levels of learning exist and attempt to design a course that will explicitly promote student development with respect to higher level cognitive skills such as analysis and synthesis, as well as the lower level skills. By creating distinct cognitive classiﬁcations, the creators of Bloom's taxonomy hoped the taxonomy would serve as “an aid in developing a precise deﬁnition and classiﬁcation of such vaguely deﬁned terms as ‘thinking’ and ‘problem solving’” and help educators to “discern the similarities and differences among the goals of their different instructional programs” (Bloom et al.,1956: p.10). Following along this original theme, Bloom's taxonomy can serve as a useful means of deﬁning and describing cognitive learning outcomes for students.Forexample, the taxonomy hasbeen used tohelpinstructors to explore the relative merits of textbooks with respect to cognitive learning (Karns et al., 1983), the cognitive learning content of computer programming courses (Oliver et al., 2004), and the design of assessment examinations for computer science classes (Scott, 2003). Instructors involved with spreadsheet-based DSS training coulduseBloom'staxonomyinsimilarwaystoassistwithspreadsheet education. 3. The application of Bloom's taxonomy to software skills education Despite the fact that Bloom's taxonomy is well known and instruc- torsusethistooleffectivelyforabroadrangeofsubjects,thetaxonomy receives relatively little attention in the software skills or computer science education literatures. For example, no published research concerning the use of Bloom's cognitive model to support spreadsheet instructionappearstoexist.Inthepastfewyears,however,conference articles concerning the application of Bloom's taxonomy to computer programming education have started to appear. Lister (2000) expresses concern that instructors and textbook authorsintheareaofcomputerprogrammingoftenfailtotakeBloom's cognitive levels into account when designing learning strategies. Indeed, Buck and Stucki (2000) discuss topics and assignments for students in computer programming courses are often presented to studentsinawaythatisthereverseofwhatonewouldexpectbasedon Bloom's cognitive development model. For instance, they suggest that many programming instructors do not followan instructional strategy that will allow students the opportunity to build up from lower to higher skill levels by following a well structured sequence of learning exercises. On the contrary, some students receive assignments in a programmingcoursetoperformhigherlevelskillssuchasanalysisand synthesis to create a software application orcomputer program before they have had a chance to gain an adequate comprehension of each of the components required todevelopthe application. To promote more effective teaching in the area of computer programming, Buck and Stucki (2000), Lister (2000), and Lister and Leaney (2003) advocate progressive cognitive skill development based on Bloom's taxonomy. 4. Application of Bloom's taxonomy to spreadsheet-based DSS courses: educational background and setting The Bloom's taxonomy does support the design of two different spreadsheet-based DSS courses taught within Western Washington University's College of Business and Economics. One of the courses is an upper division undergraduate course for students concentrating in the area of MIS, while the other is a graduate course for MBA students. The students taking each course are different with respect to background and experience. The undergraduate MIS students have a technical orientation and tend to be traditional college age students in theirearlytwenties. The MBAstudents generallydo not havetechnical backgrounds and vary widely in age from the early twenties to ﬁfties. Both courses are elective courses that are taught once each academic year. Each class is taught over a 10 week time period with two 2 hour class sessions per week. As the MIS and MBA programs within the College of Business and Economics are relatively small, the enroll- ments for each DSS course also tend to be small. Over the past 4 years, the mean classroom enrollment for a DSS class section, undergraduate or graduate, is about 10 students. In 2002, a new instructor took over responsibility for the two courses. Each course was subsequently redesigned to incorporate Bloom's cognitive model of learning and the spreadsheet-based DSS generation software provided by Microsoft Excel and the VBA programming language. Each course was designed to help students gain a better understanding of the conceptual topics associated with Table 1 Bloom's taxonomy applied to spreadsheet-based DSS education. Cognitive level Description Example(s) related to spreadsheet-based DSS development Knowledge Ability to recall learned material, without an understanding of the associated meaning. Recall the name of a spreadsheet function or characteristic. Remember syntax for a component of program code. Comprehension Ability to understand and interpret the meaning of learned material. Understand what a formula in a spreadsheet cell will do. Review program code and predict its behavior. Application Ability to use learned material in new and concrete situations. Insert the appropriate spreadsheet formula into cell or implement program code to satisfy a system speciﬁcation. Analysis Ability to break a problem into its component parts. This may include the identiﬁcation of the parts and the relationship(s) between parts. Review and understand a spreadsheet-based DSS with the objective of enhancing functionality. Analyze the performance of program code. Debug spreadsheet formulas and program code. Synthesis Ability to put parts together to create a new entity. Involves creative behaviors and formulation of new patterns or structure. Design and create a DSS application based on the system speciﬁcations. Combine spreadsheet properties and program code to create an integrated DSS. Evaluation Ability to judge the value of material for a given purpose. Evaluate the quality of a spreadsheet-based DSS with respect to stated system speciﬁcations. Evaluate and assess the quality of information gathered for purposes of system speciﬁcation. References: Bloom et al (1956), Buck and Stucki (2000). 209C.K. Tyran / Journal of Business Research 63 (2010) 207 –216 DSS, as well as a strong set of spreadsheet-based software skills. Spreadsheet-based software skills encompass both spreadsheet skills and VBA programming skills. As the undergraduate MIS students in the College of Business and Economics have a much stronger technical background with respect to spreadsheets, database, and programming than the typical MBA student, the undergraduate DSS course is more advanced with respect to spreadsheet-based DSS skills. Since the redesign of the courses in 2002, there have been ﬁve sections of the undergraduate MIS course and four sections of the MBA course. All sections of the two courses have been taught by same instructor. While the courses have evolved somewhat since 2002 to reﬂect updates in the area of DSS, there have been very few course modiﬁcations with respect to the learning strategy based on Bloom's taxonomy or the content of spreadsheet-based software skills. This article will focus on the aspects of the courses which concern the spreadsheet-based software skills. 5. Development of learning strategy and learning activities based on Bloom's taxonomy A key goal for each of the DSS courses is to help students develop their spreadsheet-based software skills such that they could ulti- mately gain a skill set that would correspond to three of the higher levels of Bloom's cognitive model of application, analysis, and synthesis. To reach this goal, developing an appropriate learning strategyand a set of learning activities was necessary. Fig. 2 shows the different types of learning activities that were developed for the DSS courses, along with their correspondence to the different cognitive levels of Bloom's taxonomy. The learning strategy for the DSS courses was designed based on the principle that students need to master Bloom's lower level cognitive skills before moving onto higher level skills. This principle draws on the concept of a learning hierarchy (Gagne, 1970; Gagne, et al., 1988), in which the learning of new intellectual skills is greatly facilitated when a learner is able to draw on prerequisite skills that have already been mastered. Hence, a key task for instructional design is to determine an appropriate sequence of prerequisite skills that will lead to the desired learning outcome. Bloom's taxonomy can provide a useful framework for developing a sequence of instructional learning activities that will support learning. As Bloom et al. (1956) note, while the taxonomy is not necessarily a strict hierarchy, the taxonomy does “represent something of a hierarchical order of the different classes of objectives. The objectives in one class [i.e., taxonomy level] are likely to make use of and be built on the behaviors found in the preceding classes. In the case of the DSS course, Bloom's taxonomy offers insight into the sequence of skills that may be needed to gain a high level of learning. For instance, before a student can use a spreadsheet formula to accomplish a speciﬁc task at the application level, it is necessary to understand what the formula will do at the comprehension level. Similarly, before a learner can examine and enhance a given type of functionality for a spreadsheet-based DSS on the analysis level, it is important that the student understand how to apply the related skill in a more general way at the application level. A learning approach in which the student is provided an opportunity to develop skills in this systematic way can help to strengthen self-efﬁcacy and satisfaction with the learning process (Bandura, 1997; Debowski, et al., 2001). Based on the foregoing strategy, the learning activities shown in Fig. 2 were introduced in a sequence that was designed explicitly to help students develop their spreadsheet software skills in a progressive fashion. To support the development of the higher-level cognitive skills, the learning strategy involved numerous hands-on learning experiences. The extensive use of hands-on learning was based on pedagogical research in the area of computer programming education, which has found that an experience-based learning ap- proach involving the use of computer laboratories and assignments can lead to enhanced learning (ACM and IEEE, 2001; Mayer, 1981; Tucker, 1996). The software skills for each DSS course were organized into focused modules of related skills that were designed to be learned as part of a class session. Each module of skills was associated with a theme. For example, the theme of “what if’ analysis for decision Fig. 2. Relationship of learning activities to Bloom's Taxonomy. Notes: 1) Highlighted cells indicate the Bloom skill levels that are most strongly addressed by the associated learning activity. 2) Comprehensive software skill projects were administered only for the undergraduate DSS course. 210 C.K. Tyran / Journal of Business Research 63 (2010) 207–216 making was associated with the development of spreadsheet forecasting models, while the theme of DSS user interface design was associated with the skills related to the development of custom user forms in VBA. The modules of focused skills for the under- graduate MIS course and the MBA course are summarized in Appendices A and B, respectively. While there was some overlap in speciﬁc skill content covered across the two courses, about 30%, the majority of the skill content covered was different across the two DSS courses. For each module of software skills, a student encountered a series of learning exercises, including readings and lecture preparation questions, hands-on tutorial exercises, classroom lecture and discus- sion, and a focused software exercise. As indicated in Fig. 2, each of these learning exercises corresponded to speciﬁc cognitive skill levels ranging from the knowledge to the application level. Additional learning activities including comprehensive skill exercises, a ﬁeld prototype project, and peer evaluation (see Fig. 2) were also assigned to allow students an opportunity to leverage their foundation set of skills into higher-level skills. Starting with the activities aimed at developing lower-level cognitive skills (see Fig. 2), each of the different types of learning activities created for the DSS courses is discussed below. To supplement this discussion, Table 2 summarizes the learning activities forthe undergraduate andMBA studentgroupsandindicates theways in which the learning activities were different across the class groups. • Readings in software textbook and preparation questions: Textbook readings were used to support the students' initial development of cognitive spreadsheet software skills relating to the levels of understanding and comprehension. Instructors selected the soft- ware textbooks used for each DSS course based on the coverage of material and availability of learning exercises that would address these cognitive levels along with the application level. Hence, when selecting course textbooks, a book's congruence with Bloom's cognitive learning model was explicitly considered. A textbook by Zak (2001) which focused on VBA skills was selected for the undergraduate MIS course, while a book by Friedrichsen (2002) that covered advanced spreadsheet skills and VBA was used for the MBA course. To encourage students to review key knowledge and comprehension areas, the instructor prepared a set of preparation questions associated with the readings for each skills module. Students were informed that they should be ready to discuss the preparation questions during class lecture. • Tutorial exercises in software textbook: Each of the software text- books included structured and detailed hands-on tutorial exercises that provided students with an opportunity to develop their cognitive skills with respect to the application level. All students were expected to work through the assigned tutorial exercise prior to the class lecture. This step was considered to be very important, since it gave students an opportunity to determine whether or not they had actually developed their module-related skills at the understanding and comprehension levels. Typically, as one would predict from Bloom's cognitive model, if students had not adequately developed related skills at the lower cognitive levels, this would become evident when they attempted to advance their skills to the application level. Students were not required to turn in the tutorial exercises for credit. • Classroom lectures on software skills: Instructors used each classroom lecture to review and clarify skills associated with the knowledge and comprehension levels for the skill module. For example, lecture topics addressed questions such as “What is a PivotTable and why is it useful for decision making?”, “What is the syntax associated with a repetition loop in VBA?”, and “What is a custom form and why would a user interface form be useful for a DSS?” To assess the state of the students' learning, the instructor solicited questions from students and orally quizzed them regarding the lecture preparation questions. The lecturetimewas relativelyshort andfocused on areas in which students needed additional help, as the instructor attempted to ensure that students had gained a solid comfort level with respect to their foundation knowledge and comprehension prior to beginning the focused software skill exercise. • Focused software skill exercises: While the software textbooks were useful for helping students create a solid foundation of cognitive skills, in the view of the instructor they generally did not provide hands-on materials that were robust enough to fully develop a student's application skill level. As a result, the instructor created focused software skill exercises for each skill module to provide students with additional application-based skills experience. The Table 2 Learning activities for spreadsheet-based DSS education. Learning activities Description of learning activity Way(s) that MBA class differed from MIS class Undergraduate MIS class MBA class Readings in software textbook & preparation questions The readings and preparation questions addressed DSS conceptual and skill building topics. A new reading assignment was assigned for each regular class day. Focus for entire course was on application of Visual Basic for Applications (VBA) programming language to spreadsheet-based DSS. Approach to learning activity was same as the undergraduate MIS class. Due to limited technical background of MBA students, the focus for ﬁrst 60% of MBA course was on advanced spreadsheet skills; focus for remainder was the pplication of VBA to spreadsheet-based DSS. Tutorial exercises in software textbook A hands-on tutorial accompanied each reading assignment. Approach to learning activity was same as the undergraduate MIS class. A different textbook (with extra focus on advanced spreadsheet skills) was used for MBA class. Classroom lectures on software skills A brief lecture and question and answer session were provided during class time. Approach to learning activity was same as the undergraduate MIS class. Lectures covered a different set of skills (see above). Focused software skill exercises Ten exercises, with each exercise corresponding to one of the ten modules listed in Appendix A. All exercises involved the application of VBA. Ten exercises, with each exercise corresponding to one of the ten modules listed in Appendix B. The ﬁrst six exercises involved advanced spreadsheet skills for DSS development; ﬁnal four exercises involved the application of VBA. Comprehensive software skill projects Two comprehensive projects were assigned. These exercises were assigned after the completion of the third and the seventh “focused skills” exercises. None. No comprehensive projects. (To bolster database knowledge, database readings and skills exercises were assigned instead.) Field project: DSS prototype Each student identiﬁed a practical application for a DSS and created a prototype system using principles and skills learned in the class. Each prototype system was formally presented to the class. Project duration was ﬁve weeks. Approach to learning activity was same as the undergraduate MIS class. Prototypes for MBA class involved less emphasis on VBA applications. Peer review of student prototype presentations Each student prepared a peer review of each classmate's prototype based on the prototype presentations. Learning activity was same as the undergraduate MIS class. None. 211C.K. Tyran / Journal of Business Research 63 (2010) 207 –216 instructor built these exercises around a spreadsheet-based decision making application and employed a somewhat less structured style than that of the tutorials in the textbooks. The skills exercises required students to work more independently when applying their skills. Also, these exercises offered students an opportunity to extend their application abilities by applying their skills in contexts that were different from the textbook readings and tutorials. Students started all of the focused software skill exercises during class time. The classroom had laptop computers for all students to provide them with a chance to consult easily with the instructor on a one-on-one basis regarding any remaining concerns related to their understanding, comprehension, or application of the software skills. In this way students were able to ﬁrm up their lower level cognitive skills before moving forward. Approximately 50% of each class period was allocated for independent work time on the exercises. Students began each focused software skill exercise during class time, but the exercises were too lengthy to be completed during class. Instead, students ﬁnished the exercises outside of class and submitted via e- mail within 24 h s of the end of class. Since high-quality feedback is an important element of the learning process (Gagne, 1984), the instructorevaluated and graded all exercise submissions and provided detailed feedback to each student by the next class session. In this way, the instructor provided students with feedback information to help them build their skills prior to the start of the next focused skills exercise. The undergraduate DSS class completed ten spreadsheet software skills exercises, with each exercise corresponding to one of the ten modules listed in Appendix A. AsAppendix A shows, the under- graduate class immediately began with exercises concerning the applications of the VBA programming language to spreadsheet-based DSS development. The MBA class also completed 10 focused spreadsheet software skills exercises. As indicated in Appendix B, the ﬁrst six modules for the MBA class concerned advanced spreadsheet skills for DSS development, while the ﬁnal four modules concerned the application of VBA to spreadsheet-based DSS develop- ment. The instructordesigned different skill exercises for the two class groups due to the differences in the students' technical background. The undergraduate MIS students had more experience with spread- sheet development and programming, while the MBA students generally had less spreadsheet experience and no programming background. • Comprehensive software skill exercises: For the undergraduate course we used two comprehensive exercises. We assigned these exercises after the completion of the third and the seventh focused skills exercises. These exercises were more complex and less structured than the focused skill exercises and were designed to further extend application skills and promote the developmentof analysisskills. On the class period before the due date for the comprehensive assignments the instructor allocated class time to allow students to work on the comprehensive exercises and seek support. All comprehensive exercises were individual assignments and were graded. As with the focused exercises, instructor feedback was provided by the start of the following class period. The comprehen- sive assignments include the development of DSS applications such as a “Retirement Investment Planner Support System” and a “Data Retrieval and Plotting System” (see Fig. 1). The MBA students did not do the comprehensive skills exercises involving spreadsheet-based DSS development. The reason for this was that the MBA students generally did not have a strong background in database. Since database concepts are important with regard to the topic of DSS, the instructor assigned database- related readings and focused database skills exercises in lieu of the comprehensive exercises. While comprehensive exercises on spreadsheet development would have been useful for the MBA students, we felt that it was even more useful for the students to gain a fundamental understanding of database concepts so that they would be able to understand better the role of the database component of a DSS. • Field project – DSS prototype application: The instructor created the DSS prototype application assignment to provide students in the undergraduate and MBA classes with an opportunity to work on an unstructured project that would extend their cognitive skills with respect to Bloom's skill levels of analysis and synthesis. For the project assignment, we asked students to create a working spreadsheet-based DSS prototype application that would be useful fora real world decision maker. We gave studentsmuch leewaywith respect to choosing a project. The key criteria for project selection were to ﬁnd a project that would support decision making, would involve the use of spreadsheet-based and VBA skills gained in the course, and would be interesting to the student. We assigned the project during the fourth week of the ten week academic term so that students would be able to choose a project application after gaining a solid orientation concerning DSS concepts and skills during the ﬁrst month of the class. To reduce the chances for procrastinationwith respecttoproject choiceand DSS development, we required students to prepare and review a project proposal with the instructor by the sixth week of the academic term, and to provide a hands-on status update at the beginning of the eighth week of the quarter. While the assignment includes the same set of project speciﬁcations to the undergraduate and MBA classes, the prototypes for the undergraduate class use more VBA-based components, since the undergraduate class covered more VBA-related skills. Examples of student DSS prototype applications included a DSS to support mutual fund investments using an efﬁcient frontier model, a DSS to support ﬁnancial planning for a real estate development ﬁrm, and a DSS to support employee scheduling for a retail store. As students worked on the prototypes, the instructor typically interacted with the students as they gained experience with Bloom's skill levels of analysis and synthesis. In addition to ofﬁce hours, we allocated class timeforone-on-one progress report sessionsbetweenthe instructor and students. The instructor evaluated each DSS prototype applica- tion based on the project guidelines. • Peer review of student prototype presentations: During the ﬁnal week of the DSS course, each student delivered a presentation and demonstration of his or her DSS prototype application to the entire class. Basedon thesepresentations,werequired eachof thestudents in the audience to prepare and submit a short hand-written review of each prototype application developed by a classmate. We asked students to assess the quality of each prototype application with respect to the stated project guidelines. The peer review exercise was the same for undergraduate and MBA courses, implemented to provide students with an opportunity to gain experience evaluating DSS applications on Bloom's evaluation cognitive skill level. The instructorevaluated studentpeerreviewswithrespecttothe quality of assessment. We provided copies of the peer reviews to the students so that they could get feedback from their peers regarding the prototype projects. 6. Survey study and assessment of teaching approach This section reports the results of a student survey for each DSS course. The survey data are descriptive and focuses on the students' perceptions of learning outcomes and activities associated with the Bloom taxonomy-based learning strategy described above. We collected survey data for all sections of the undergraduate and graduate DSS courses over a four year period spanning spring 2002 through winter 2006. During this period we taught ﬁve sections of the undergraduate DSS course with a mean enrollment of 9 students. We delivered four sections of the graduate MBA DSS course with a mean enrollment of 11 students. 212 C.K. Tyran / Journal of Business Research 63 (2010) 207–216 The collected survey data include student backgrounds on the ﬁrst day of class and at the end of the course. For purposes of reporting, the study aggregated the survey data into two groupings, the under- graduate MIS course and the graduate MBA course. Although survey data were collected over an extended period of time, an analysis revealed that the survey ﬁndings for each DSS course tended to be stable across class sections for each course. The studyapplied the non- parametric Kruskall Wallis test for statistical differences across the class sections. For the undergraduate sample the comparison involved the ﬁve undergraduate class sections (n=45), and for the MBA sample the comparison involved the four MBA class sections (n=43). In no case was a statistical difference found among class sections. Hence, this report focuses on the survey data in an aggregate form for each course. 7. Pre-survey of student skills On the ﬁrst day of each class we conducted a pre-survey to learn more about the students' backgrounds and perceived skill levels at the beginning of the class. Due to the differences in technical backgrounds of the MIS and MBA students, some of the pre-survey questions were different for each student group. Prior to taking the DSS course, the undergraduate MIS students had successfully completed two pre- requisite courses that explicitly involved the learning of spreadsheet skills in a computer lab as well as a programming course in Visual Basic. As we expected, the MIS students had strong spreadsheet skills, andthe instructor's observations conﬁrmed this. The skills component of the pre-survey for the MIS students focused on Visual Basic pro- gramming skills, rather than spreadsheet skills. The skills component of the pre-survey for the MBA students focused on spreadsheet skills, since only three of the 43 MBA students reported having a computer programming course, and all typically had less background in spreadsheet skills. Table 3 summarizes student perceptions regarding their technical backgrounds entering the DSS course. The pre-survey for the students in the MIS undergraduate group focused on gaining an assessment of their application programming skills. The survey data indicated that the students believed that they entered the course with solid Visual Basic programming skills, which we expected since a Visual Basic course was a prerequisite for the DSS course. The students thought, however, that they had only superﬁcial knowledge of VBA. Few of the MIS students indicated that they were familiar with using macros or VBA to create a spreadsheet-based DSS application. Also, the undergraduate MIS students did not seem to have a strong knowledge of DSS, as they indicated low familiarity with the components of DSS and a “what if” DSS spreadsheet application. The ﬁrst-day survey for MBA students focused on Excel spread- sheet skills. The survey drew on a spreadsheet skill rating system developed by Pemberton and Robson (1995) to gauge the students' spreadsheet expertise. Pemberton and Robson (1995) identiﬁed eight levels of spreadsheet skills, which included cell formatting at the lowest level, basic business formulas at level 3, advanced/specialized functions at level 7, and VBA macros and user interfaces at level 8, the highest level. The survey for the MBA students included sample questions to assess experience with respect to Excel skills at these levels. The survey ﬁndings shown in Table 3 indicate that the MBA students generally considered themselves to have strong skills with respect to the lower level types of spreadsheet skills, such as format- ting and basic business formulas. However, the MBA students' knowledge of more advanced Excel functions such as LOOKUP and PivotTables (Microsoft Ofﬁce Specialist Certiﬁcation, 2006), Excel macros, and VBA tended to be superﬁcial. Based on these ﬁndings, it was clear that the MBA students had much room for growth with respect to these types of skills. 8. Student assessment of learning outcomes and activities The studyevaluates the impact of the learning strategyadopted for the DSS courses by assessing student perceptions regarding their learning performance and the learning activities. Although the use of student ratings of instruction to assess learning has been the topic of debate, meta-analytic reviews of research studies that have examined student ratings have found evidence to support the validity of student ratings (D'Appollonia and Abrami, 1997; Greenwald, 1997). In particular, a meta-analysis by Feldman (1997) ﬁnds that student ratings of perceived outcome of instruction were strongly correlated to student achievement. Also, as noted by Gagne (1984), student attitudes and perceptions are an important type of learning outcome, since attitudes can inﬂuence a student's learning behaviors and Table 3 MIS and MBA student perceptions on pre-survey: DSS, excel spreadsheet, and applications programming skills. Questionnaire item MIS class mean (Std) MBA class mean (Std) Age (years) 23.1 (2.4) 28.2 (6.7) Degree to which I am familiar with the following concept: Components of a decision support system 2.53 (0.81) 1.84 (0.86) Degree to which I am familiar with the following Microsoft Excel spreadsheet skills or Visual Basic programming skills: Using cell format features of Excel (e.g., color, font, etc) to enhance spreadsheet appearance NA 4.34 (0.83) Using basic Excel business formulas (e.g., SUM, PV) NA 3.96 (0.74) Using the “LookUp” feature in Excel NA 2.70 (1.21) Using the “PivotTable” feature in Excel NA 2.25 (1.10) Using Visual Basic to create user interface features on forms (e.g., check boxes, menu items, option buttons) 3.96 (0.74) NA Using Visual Basic to program “conditional” logic (e.g., “If-then”, “Select case” statements) 3.93 (0.56) NA Using Visual Basic to program “repetition” logic (e.g., “Do” and “For” loops) 3.89 (0.54) NA Degree to which I am familiar with the following Microsoft Excel spreadsheet application development and Visual Basic for Applications programming skills: Development of “what if” business models using Microsoft Excel 2.44 (0.94) 2.80 (1.32) Development of “macro” programs in Microsoft Excel 2.82 (0.78) 2.43 (1.26) Using Visual Basic for Applications to enhance Microsoft Ofﬁce Applications 2.59 (0.72) 1.93 (0.99) The “object model” for Microsoft Ofﬁce Applications 2.22 (0.70) NA Notes: 1) Sample size: N for MIS students=45 (out of 46 enrolled students); N for MBA students=43 (out of 43 enrolled students). 2) Questionnaire items scaled from 1 to 5 as follows: 1- I have no knowledge about the skill; 2- I have heard about the skill but could not discuss or demonstrate the skill; 3- I could discuss or demonstrate the skill in a superﬁcial way with someone else; 4- I could discuss or demonstrate the skill in detail with someone else; 5- I consider myself a true expert with regard to this skill. 3) “NA” means that the question was not posed to the sample group. 213C.K. Tyran / Journal of Business Research 63 (2010) 207 –216 reactions to a learning experience. Given these ﬁndings from the research literature, we determined that the students' perceptions regarding their learning outcomes would provide a relevant measure of the effectiveness of the instructional approach used for the DSS courses. To capture student feedback, we administered an anonymous surveyon the ﬁnal day of class. Tables 4 and 5 summarize the ﬁndings. Table 4 summarizes the post-survey ﬁndings regarding student perceptions of learning. With regard to the undergraduate MIS course, the survey results strongly indicate that the students thought that they had gained a better understanding of DSS and its components after taking the class, as all ratings were above 6 on a 7-point scale. In addition, the students indicated that they felt very conﬁdent they could develop DSS using their VBA development skills. Given that the presurvey indicated most students began the term with no more than a superﬁcial understanding of DSS or applications programming with VBA, these ﬁndings indicated the students felt they had greatly improved their skills in the area of DSS development. A one sample t test analysis was conducted to evaluate whether or not the students' postsurvey perceptions regarding learning were signiﬁcantly greater than a threshold test value for perceived learning. The items on the postsurvey instrument assessed perceived learningon a 7-point scale ranging from 1 (strongly disagree), to 4 (neutral), to 7 (stronglyagree). For the purposes of the onesample t test, avalue of 5.5 on the 7-point survey scale was chosen to serve as the threshold test value, as this value is halfway between the ratings of neutral and strongly agree and represents a relatively high threshold value for perceivedlearning. As indicatedbythet testresults inTable 4,themean values for perceived learning on the postsurvey were signiﬁcantly greater than the test value of 5.5 at a very high level of conﬁdence (pb.001) for all of the items for the undergraduate MIS course. As indicated inTable 5, separate survey items asked the students to assess the usefulness of the textbook, lectures, focused exercises, comprehensive exercises, and feedback. The students indicated that all of these aspects of the course were useful, with the hands-on focused and comprehensive exercises being particularly worthwhile. Also, it is clear that the students felt that using class time to start the hands-on exercises was very helpful. The students indicated that they did not consider the course to be easy. The perceptions of the MBA students regarding their learning outcomes and the learning activities were also very positive. The numerical ratings displayed in Table 4 indicate that the MBA students had a favorable impression with respect to the improvement of their spreadsheet-based development skills, and that they felt conﬁdent that they could develop a DSS using the skills learned in the course. The one sample t test ﬁndings relating to the perceived learning measures in Table 4 shows that the MBA students' ratings were Table 4 MIS and MBA student perceptions on post-survey: learning and development skills related to decision support systems development. Questionnaire item MIS class mean (Std) MBA class mean (Std) Today I have a better understanding of what a “decision support system” (DSS) is than I did ten weeks ago. 6.72⁎(0.54) 6.63⁎(0.62) Through the hands-on exercises and homework assignments, I gained a better understanding of how: a developer might use spreadsheet software (e.g., Excel) to help a decision maker create a “What If” model to support decision making NA 6.66⁎(0.53) a developer might use spreadsheet software (e.g., Excel) to help a decision maker analyze data to support decision making. NA 6.56⁎(0.59) a developer might create an effective and easy-to-use user interface for a DSS. 6.17⁎(0.96) 6.59⁎(0.81) a database may be used to support decision making with a DSS. 6.11⁎(0.87) 6.32⁎(1.06) With the experiences that I have gained via this course, I believe that: my skills with the Excel spreadsheet software have improved NA 6.78⁎(0.42) I have developed a better appreciation for what is involved in developing program code with the VBA programming language NA 6.33⁎(1.31) Today I am more conﬁdent in my abilities to develop a VBA and Microsoft Ofﬁce based DSS (i.e., a system that incorporates the user interface, model and data components) then I was ten weeks ago. 6.51⁎(0.69) 6.59⁎(0.81) If I had a job where I was asked to implement a DSS using VBA and Microsoft Ofﬁce applications that involved the development of a) a professional and easy-to-use user interface, b) a simple business model component, and c) data retrieval from a data base, I am conﬁdent that I have the skills to develop this type of DSS. 6.22⁎(0.64) 5.89 (0.98) Notes: 1) Sample sizes: N for MIS students=46 (out of 46 enrolled students); N for MBA students=41 (out of 43 enrolled). 2) Questionnaire items scaled from 1 (strongly disagree) to 4 (neutral) to 7 (strongly agree). 3) “DSS” refers to “Decision Support Systems”; “VBA” refers to “Visual Basic for Applications” programming language; “NA” means that the question was not posed to the sample group. 4) A one-sample t-test analysis using a threshold value of 5.5 (a value halfway between a rating of 4 (“neutral”) and 7 (“strongly agree”)) was conducted on the survey ﬁndings shown above. A “⁎” indicates that the ﬁndings from the one-sample t-test were statistically signiﬁcant at the pb.001 level. Table 5 MIS and MBA student perceptions on post-survey: course components used to aid learning of decision support systems. Questionnaire item MIS class mean (Std) MBA class mean (Std) Value of course learning activities: The textbook that we used for the technical aspects of this course was useful and I would recommend that the book be used again. 5.62 (1.34) 5.71 (1.60) The class lectures related to technical skills helped me to better understand the concepts that we needed to apply for the “skill building” hands-on exercises. 5.49 (1.32) 5.28 (1.32) The process of working on the focused “skill building” hands-on exercises helped me to develop my DSS development capabilities. 6.28 (1.10) 6.37 (0.80) The process of working on the comprehensive homework assignments helped me to develop my spreadsheet-based DSS and Visual Basic for Applications (VBA) programming capabilities. 6.41 (0.91) NA Implementation of course learning activities: We did too many hands-on exercises in this class. 2.15 (1.40) 2.66 (1.78) I would have preferred not to start the “skill building” hands-on exercises in class. I would have preferred doing the exercises strictly outside of class. 1.91 (1.32) 1.93 (1.62) The amount (and detail) of feedback that I got on the hands-on exercises was sufﬁcient. 5.89 (1.22) 5.59 (1.45) I would have preferred that this course be more of a traditional lecture course and has less of a hands-on system development component. 1.68 (1.53) 1.38 (0.77) This class was not challenging enough for me. 2.72 (1.36) 2.51 (1.50) Notes: 1) Sample sizes: N for MIS students=46 (out of 46 enrolled students); N for MBA students=41 (out of 43 enrolled). 2) Questionnaire items scaled from 1 (strongly disagree) to 7 (strongly agree). 3) “DSS” refers to “Decision Support Systems”; “VBA” refers to “Visual Basic for Applications” programming language; “NA” means that the question was not posed to the sample group. 214 C.K. Tyran / Journal of Business Research 63 (2010) 207–216 signiﬁcantly greater than the test value of 5.5 (pb.001) for all but one of the survey items. The one item that was not statistically different from a value of 5.5, an item concerning comprehensive DSS develop- ment skills, was still rated highly by the MBA students with a mean of 5.89 on the 7-point scale. Table 5 shows that the MBA students had a positive view concerning several different aspects of their learning experience. As with the undergraduate course, the MBA students indicated that it was useful to include the hands-on exercises and to allocate class time to get a start on the exercises. Also, the MBA students indicated that the course was challenging. The ﬁnal two items on the student surveys were open-ended questions asking students to comment on the hands-on aspects of the course that they liked the best and areas for improvement. These comments were consistent with the ﬁndings discussed above, as students mentioned that they found the exercises to be very useful, practical, and easy to follow. Students liked the way that the learning was handled in increments. Ideas forenhancement indicated that some students would prefer to have more hands-on exercises and fewer lectures, while others felt that the workload needed to be reduced. 9. Instructor observations regarding student performance and attitudes The skills exams for the DSS courses were hands-on exams that required students to apply their skills under a controlled exam envi- ronment. We designed the exams to assess learning at the application level of cognitive skill. Due to practical considerations and low sample size, it was not feasible to establish a control group or conduct a comparative study to assess the impact of the Bloom-based learning strategy on student performance. However, the author can draw on past experiences as a teacher in the area of spreadsheet skills and computer programming to comment on the relative performance of students who were enrolled in the spreadsheet-based DSS courses discussed in this article. For the author, prior to adopting a Bloom- based teaching strategy, a recurrent problem with teaching computer skills and programming courses was that a signiﬁcant portion of students fell behind with respect to cognitive skill development and then performed poorly on hands-on exams. However, for the DSS courses described in this article, relatively few of the students in the class performed at a weak level, and a relatively high proportion performedat a strong level. Forexample,across the ﬁve sections of the undergraduate MIS course, only 3 of the 46 students had a skills exam average below 60%. This small fraction of low scores is unusual in the author's experience. A similar type of result occurred for the MBA course. Students tended to have very positive attitudes about the course and the learning exercises. Students seemed to grasp quickly the value of the learning strategy that the instructor employed, and they appeared to enjoy taking on each new course assignment. On the formal course evaluations forms distributed by the university, many students indicated that they considered the DSS course to be their favorite class. 10. Discussion and directions for future research Bloom'staxonomy isapplicabletoa spreadsheet-basedDSS course. Knowledge of the taxonomy can support an instructor's awareness of the different cognitive skill levels, provide guidance for instructional design, inform the development of learning exercises, provide a basis for the evaluation of textbooks, and help establish course educational objectives. Based on the survey results and statistical ﬁndings regarding the perceptions of learning, it appears that the learning strategy adopted for the spreadsheet-based DSS courses was well received by two distinct types of student groups: undergraduate MIS students and MBA students with a less technical background. While more research with different types of student groups is necessary to establishthegeneralizabilityof theresults of this study, the ﬁndingsin this report are encouraging. In addition to the course component issues inTable 5, other factors relating tothe course learning strategy and circumstances may also be partial explanations of the favorable student performances and attitudes. First, the learning strategy adopted for the DSS courses involved an approach inwhich students progressively developed their skills in a guided way. Research has found that this type of guided learning approach can promote self-efﬁcacy, learner satisfaction, and achievement(e.g., Debowskiet al.,2001). Second,an important aspect of the learning strategy used for the DSS courses was that the students worked on active learning activities in which they had a degree of control during the learning process, for example, in the skills exercises and the prototype application project. Learner control can promote favorable attitudes toward instruction (Fry, 1972) and learning engagement (Fisher et al., 1975). In combination with appropriate instructional support, learner control enhances learning (Stipek and Weisz, 1981). Hence, the factor of learner control may have also helped to contribute to the students' positive perceptions regarding the learning experiences and outcomes in the DSS courses. Third, the studentsmayhave naturallyself-selectedintothe elective DSS courses due to their interest or aptitude for the topic. As such, students may have had predispositions to view the course in a positive way. Looking to the future, numerous avenues are available for research concerning the application of Bloom's taxonomy for the spreadsheet- based DSS course. For example, extending the ﬁndings by exploring other ways to implement learning strategies for spreadsheet-based learning based on Bloom's cognitive model would be worthwhile. Also, Bloom's taxonomy revisions incorporate a two-dimensional structure (Krathwohl, 2002). Research that investigates how the revised taxonomy might be applied to spreadsheet-based DSS edu- cation would be useful. Other interesting research could include the assessment of the impact of different learning strategies on student skill performance, the assessment of the effect of variations in learner guidance and control on student perceptions and performance, and comparative empirical research that uses objective learning perfor- mance as an outcome measure. Given the importance of spreadsheet- based decision support systems in organizations, ongoing research into ways to enhance spreadsheet education is justiﬁable. Appendix A Set of focused spreadsheet-based skill modules for undergraduate MIS course • Introduction to Visual Basic for Applications (VBA) and the Visual Basic Editor • Introduction to the VBA Object Model • VBA and object variables — using object variables to manipulate objects and properties • Using VBA to work with string and date variables • Using VBA to work with numeric variables • Creating a high quality DSS user interface using custom forms with VBA • Retrieving data from a database into a spreadsheet • Using VBA to manipulate data within a spreadsheet • Using VBA to plot data into charts • Using automation to integrate DSS applications Appendix B Set of focused spreadsheet-based skill modules for graduate MBA course • Using a spreadsheet to develop “What If” types of DSS • Using a spreadsheet to perform statistical correlation and regression analysis 215C.K. Tyran / Journal of Business Research 63 (2010) 207 –216 • Using a spreadsheet to manage a list of information • Using a spreadsheet to ﬁlter and sort data for decision making • Using a spreadsheet to interactively evaluate data using PivotTables and PivotCharts • Supporting decision making by using advanced ﬁnancial, database and lookup spreadsheet functions • Introduction to Visual Basic for Applications (VBA) and the Visual Basic Editor • Introduction to the VBA Object Model • Creating a high quality DSS user interface using custom forms with VBA — introductory skills • Creating a high quality DSS user interface using custom forms with VBA — intermediate skills References Ainsworth P. Restructuring the introductory accounting courses: the Kansas State University experience. J Account Educ 1994;12(4):305–23. Anderson LW, Sosniak LA, editors. Bloom's taxonomy: a forty year retrospective. Chicago, IL: National Society for the Study of Education; 1994. Association of Computing Machinery/Institute of Electrical and Electronics Engineers. Computing curricula 2001: Report of the ACM/IEEE. Computer Society Press; 2001. Athanassiou N, McNett JM, Harvey C. Critical thinking in the management classroom: Bloom's taxonomy as a learning tool. J Manag Educ 2003;27(5):533–55. Bandura A. Self-efﬁcacy: the exercise of control. New York: Freeman; 1997. Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomy of educational objectives: the classiﬁcation of educational goals. New York: Longman; 1956. Buck D, Stucki DJ. Design early considered harmful: graduated exposure to complexity and structure based on levels of cognitive development. Proceedings of the 31st SIGCSE Symposium on Computer Science Education, Austin, TX, 75–79; 2000. Chyung S, Stepich D. Applying the ‘congruence’ principle of Bloom's taxonomy to designing online instruction. Q Rev Distance Educ 2003;4(3):317–30. D'Appollonia S, Abrami PC. Navigating student ratings of instruction. Am Psychol 1997;52(11):1198–208. Davis G, Naumann J, Allen G. Developing and teaching IS'97.2: Personal productivity with information technology. Commun AIS 1999;1(17). Debowski S, Wood RE, Bandura A. Impact of guided exploration and enactive exploration on self-regulatory mechanisms and information acquisition through electronic search. J Appl Psychol 2001;86(6):1129–41. Feldman KA. Identifying exemplary teachers and teaching: evidence from student ratings. In: Perry RP, Smart JC, editors. Effectiveteaching in higher education. Bronx, NY: Agathon Press; 1997. p. 368–95. Fisher MD, Blackwell LR, Garcia AB, Greene JC. Effects of student control and choice on engagement in a CAI arithmetic task in a low-income school. J Educ Psychol 1975;67(6):776–83. Friedrichsen L. Microsoft Excel 2002 with Visual Basic for Applications. Cambridge, MA: Course Technology; 2002. Fry JP. Interactive relationship between inquisitiveness and student control of instruction. J Educ Psychol 1972;63(5):459–65. Gagne RM. The conditions of learning. (2nd ed.). New York: Holt, Rinehart, and Winston; 1970. Gagne RM. Learning outcomes and their effects: useful categories of human performance. Am Psychol 1984;39(4):377–85. Gagne RM, Briggs LJ, Wager WW. Principles of instructional design. (3rd ed). Fort Worth: Holt, Rinehart, and Winston; 1988. Greenwald AG. Validity concerns and usefulness of student ratings of instruction. Am Psychol 1997;52(11):1182–6. Hamblen KA. An art criticism questioning strategy within the framework of Bloom's taxonomy. Stud Art Educ 1984;26(1):41–50. Karns J, Burton G, Martin G. Learning objectives and testing: an analysis of six principles of economics textbooks, using Bloom's taxonomy. J Econ Educ 1983;14(3):16–9. Keen PGW, Scott Morton MS. Decision support systems: an organizational perspective. Reading, MA: Addison-Wesley; 1978. Krathwohl DR. A revision of Bloom's taxonomy: an overview. Theory Pract 2002;41(4): 212–8. Lister R. On blooming ﬁrst year programming, and its blooming assessment. Proceedings of the 4th Australasian Computing Education Conference (ACE2000), Melbourne, Australia; 2000. p. 158–62. Lister R, Leaney J. Introductory programming, criterion-referencing, and Bloom. Proceedings of the 34th SIGCSE Symposium on Computer Science Education, Reno, NV; 2003. p. 75–9. Lovell-Troy LA. Teaching techniques for instructional goals: a partial review of the literature. Teach Sociol 1989;17(1):28–37. Marakas GM. Decision support systems in the 21st century. (2nd ed.). Upper Saddle River, NJ: Prentice Hall; 2003. Mayer RE. The psychology of how novices learn computer programming. Comput Surv 1981;13(1):121–41. Microsoft Ofﬁce Specialist Exam Skill Standards: Excel 2003 Expert webpage Retrieved September 6, 2006, from http://www.microsoft.com/learning/mcp/ofﬁcespecialist/ objectives/Excel2003ExpertExamSkillStandards. Oliver D, Dobele T, Greber M, Roberts T. This course has a Bloom rating of 3.9. Proceedings of the 6th Australasian Computing Education Conference (ACE2004), Dunedin, New Zealand; 2004. p. 227–31. Palocsay SW, Markham IS. Teaching spreadsheet-based decision support systems with Visual Basic for Applications. Inf Technol Learn Perform J 2002;20(1):27–35. Pemberton J, Robson A. The spreadsheet: just another ﬁling cabinet? Manage Decis 1995;33(8):30–5. Power DJ. Decision support systems: concepts and resources for managers. Westport, CT: Quorum Books; 2002. Pungente MD, Badger RA. Teaching introductory organic chemistry: ‘blooming’ beyond a simple taxonomy. J Chem Educ 2003;80(7):779–84. Ragsdale CT, Power DJ, Bergey PK. AMCIS 2002 panels and workshops II: spreadsheet- based DSS curriculum issues. Commun – Assoc Inf Syst 2002;9:356–65. Scott T. Bloom's taxonomy applied to testing in computer science classes. Consortium for Computing Sciences in Colleges (Rocky Mountain Conference); 2003. p. 267–74. Stipek DJ, Weisz JR. Perceived personal control and academic achievement. Rev Educ Res 1981;51(1):101–37. Tucker A. Strategic directions in computing science education. Comput Surv 1996;28(4): 836–45. Wehrs W. An applied DSS course using Excel and VBA: IS and/or MIS? Proceedings of the17th Annual Information Systems Education Conference(ISECON), Philadelphia, PA; 2000. Zak D. Visual Basic for Applications. Cambridge, MA: Course Technology; 2001. Zobel C, Ragsdale C, Rees L. Teaching decision support systems using VBA. OR/MS Today 2000;12:28–31. 216 C.K. Tyran / Journal of Business Research 63 (2010) 207–216 xemplary teachers and teaching: evidence from student ratings. In: Perry RP, Smart JC, editors. Effectiveteaching in higher education. Bronx, NY: Agathon Press; 1997. p. 368–95. Fisher MD, Blackwell LR, Garcia AB, Greene JC. Effects of student control and choice on engagement in a CAI arithmetic task in a low-income school. J Educ Psychol 1975;67(6):776–83. Friedrichsen L. Microsoft Excel 2002 with Visual Basic for Applications. Cambridge, MA: Course Technology; 2002. Fry JP. Interactive relationship between inquisitiveness and student control of instruction. J Educ Psychol 1972;63(5):459–65. Gagne RM. The conditions of learning. (2nd ed.). New York: Holt, Rinehart, and Winston; 1970. Gagne RM. Learning outcomes and their effects: useful categories of human performance. Am Psychol 1984;39(4):377–85. Gagne RM, Briggs LJ, Wager WW. Principles of instructional design. (3rd ed). Fort Worth: Holt, Rinehart, and Winston; 1988. Greenwald AG. Validity concerns and usefulness of student ratings of instruction. Am Psychol 1997;52(11):1182–6. Hamblen KA. An art criticism questioning strategy within the framework of Bloom's taxonomy. Stud Art Educ 1984;26(1):41–50. Karns J, Burton G, Martin G. Learning objectives and testing: an analysis of six principles of economics textbooks, using Bloom's taxonomy. J Econ Educ 1983;14(3):16–9. Keen PGW, Scott Morton MS. Decision support systems: an organizational perspective. Reading, MA: Addison-Wesley; 1978. Krathwohl DR. A revision of Bloom's taxonomy: an overview. Theory Pract 2002;41(4): 212–8. Lister R. On blooming ﬁrst year programming, and its blooming assessment. Proceedings of the 4th Australasian Computing Education Conference (ACE2000), Melbourne, Australia; 2000. p. 158–62. Lister R, Leaney J. Introductory programming, criterion-referencing, and Bloom. Proceedings of the 34th SIGCSE Symposium on Computer Science Education, Reno, NV; 2003. p. 75–9. Lovell-Troy LA. Teaching techniques for instructional goals: a partial review of the literature. Teach Sociol 1989;17(1):28–37. Marakas GM. Decision support systems in the 21st century. (2nd ed.). Upper Saddle River, NJ: Prentice Hall; 2003. Mayer RE. The psychology of how novices learn computer programming. Comput Surv 1981;13(1):121–41. Microsoft Ofﬁce Specialist Exam Skill Standards: Excel 2003 Expert webpage Retrieved September 6, 2006, from http://www.microsoft.com/learning/mcp/ofﬁcespecialist/ objectives/Excel2003ExpertExamSkillStandards. Oliver D, Dobele T, Greber M, Roberts T. This course has a Bloom rating of 3.9. Proceedings of the 6th Australasian Computing Education Conference (ACE2004), Dunedin, New Zealand; 2004. p. 227–31. Palocsay SW, Markham IS. Teaching spreadsheet-based decision support systems with Visual Basic for Applications. Inf Technol Learn Perform J 2002;20(1):27–35. Pemberton J, Robson A. The spreadsheet: just another ﬁling cabinet? Manage Decis 1995;33(8):30–5. Power DJ. Decision support systems: concepts and resources for managers. Westport, CT: Quorum Books; 2002. Pungente MD, Badger RA. Teaching introductory organic chemistry: ‘blooming’ beyond a simple taxonomy. J Chem Educ 2003;80(7):779–84. Ragsdale CT, Power DJ, Bergey PK. AMCIS 2002 panels and workshops II: spreadsheet- based DSS curriculum issues. Commun – Assoc Inf Syst 2002;9:356–65. Scott T. Bloom's taxonomy applied to testing in computer science classes. Consortium for Computing Sciences in Colleges (Rocky Mountain Conference); 2003. p. 267–74. Stipek DJ, Weisz JR. Perceived personal control and academic achievement. Rev Educ Res 1981;51(1):101–37. Tucker A. Strategic directions in computing science education. Comput Surv 1996;28(4): 836–45. Wehrs W. An applied DSS course using