Boser, 'The Development of Problem Solving Capabilities in Pre-service Technology Teacher Education' URL = http://hegel.lib.ncsu.edu/stacks/serials/jot/jot-v4n02-boser-the Journal of Technology Education Volume 4, Number 2 Spring 1993 ARTICLES The Development of Problem Solving Capabili- ties in Pre-service Technology Teacher Education Richard A. Boser Enhancing the problem solving capabili- ties of students and employees has become a national educational issue. The Commission on Pre-College Education in Mathematics, Sci- ence and Technology (1983) declared that "problem-solving skills, and scientific and technological literacy -- [are] the thinking tools that allow us to understand the techno- logical world around us" (p. v). More recent reports that have focused on entry-level workplace skills by Carnevale, Gainer, and Meltzer (1990) and United States Department of Labor (1991) [SCANS Report] also under- score the importance of developing students' problem solving abilities. As a result of this decade of emphasis on problem solving, efforts to enhance the capabilities of stu- dents to solve problems have reached most disciplines and most educational levels (Birch, 1986; Bransford, Goin, Hasselbring, Kinzer, Sherwood, & Williams, 1986; Kulm, 1990; Lombard, Konicek, & Schultz, 1985; Thomas & Englund, 1990). In technology education, teaching through problem solving methodology has be- come a central focus of instructional activ- ity (Waetjen, 1989). It follows, therefore, that teachers need to be adept at using prob- lem solving strategies in their classrooms and laboratories. Several recent studies highlight this need. Barnes (1987) concluded that problem solving should be a key descrip- tor for defining technology and a curricular organizer for the study of technology. Householder and Boser (1991) reported that an emphasis on problem solving instructional strategies was a key ingredient in assessing the effective implementation of pre-service technology teacher education programs. In addition, research by Horath (1990) and by Householder and Boser pointed to the need for graduates of technology teacher education programs to use problem solving strategies in their classrooms and laboratories and to teach problem solving skills. In spite of the need to implement effective problem solv- ing instruction in pre-service technology teacher education, there is no generally ac- cepted framework to guide curriculum develop- ment or assessment procedures. BACKGROUND IDEAS One difficulty in investigating problem solving behavior is the many usages of the phrase "problem solving". McCormick (1990) noted that, depending on the context, "prob- lem solving" may mean: (a) a teaching method that encourages active learning, (b) a ge- neric ability to deal with problem situ- ations, (c) a method used in such subjects as mathematics or science, or (d) an empirical investigation. Additionally, Gagne (1985) used the term problem solving to describe a higher-order intellectual ability and a way of learning. All of these usages have implications within technology teacher education. Problem solving is clearly seen as a teaching method with links back to experiential learning. Problem solving may also be viewed as a way of learning that generates new insights and useful thinking processes for the learner (Gagne, 1985). Further, the scientific method of hypothesis generating and testing is certainly at the heart of technological problem solving. In this study, because of the implications for teacher education, "problem solving" was limited to two usages. First, "technological problem solving" refers to the systematic way of investigating a sit- uation and implementing solutions. Second, the "problem solving approach" is used to de- scribe a teaching method that encourages the development of new insights and useful think- ing processes through active investigative learning. TECHNOLOGICAL PROBLEM SOLVING Technological problem solving processes have been greatly influenced by the work of Dewey and Polya (Savage and Sterry, 1991). Dewey (1910) described a five step iterative process of problem solving that comprised: (a) felt difficulty, (b) clarification of the problem, (c) identification of possible sol- utions, (d) testing the suggested solutions, and (e) verification of the results. Polya (1957) proposed a heuristic process for solv- ing problems in mathematics that provided a mental guideline for action. The steps in Polya's heuristic included: (a) understanding the problem, (b) devising a plan, (c) carry- ing out the plan, and (d) looking back -- checking the results and evaluating the sol- ution. Two additional influences on technolog- ical problem solving have been the scientific method and the idea of creative problem solv- ing. de Bono (1990) postulated that the con- cept of the "hypothesis", which formally sanctions creativity and imagination, has been "the" idea that has powered rapid scien- tific and technological change. Wallas (1926) described the creative problem solving process as involving four phases: (a) prepa- ration, (b) incubation, (c) illumination, and (d) verification. More recently, Devore, Horton, and Lawson (1989) built upon the work of Wallas and added two additional phases: motivation and manipulation. These approaches have formed the basis for many models of problem solving that have been applied in technology education. Some of these models retain the simple linear ap- proach, such as the IDEAL model of Bransford and Stein (1984), while others, such as Barnes, Wiatt, and Bowen (1990) and Hutchinson (1987), have proposed more complex circular or spiral models with evaluation components built into each phase. PROBLEM SOLVING AS AN INSTRUCTIONAL APPROACH The problem solving approach immerses students in active, investigative learning (Sellwood, 1989). Through participation in a series of practical problem solving activ- ities that may involve designing, modeling, and testing of technological solutions it is assumed that the learner will acquire both technical knowledge and higher-order cogni- tive skills. Gagne (1985) stressed the im- portance of experiential learning and noted that abstract concepts must be built upon concrete situations in order to "operationalize" (p. 103) declarative know- ledge. Andre (1986) emphasized that the im- portance of problem solving methods lies in the degree of information processing required of the learner. In identifying problems, searching for solutions, and presenting re- sults, the learner has multiple opportunities to encode and accommodate new knowledge. PREPARATION TO TEACH PROBLEM SOLVING No research was found that related training in problem solving methods to pre- service technology teacher education. How- ever, Diaber (1988) noted many common instructional elements among "investigative delivery systems" (p. 166) such as problem solving, inquiry teaching, discovery learn- ing, and critical thinking. Given the com- monalities, research in these related areas may provide useful insights to teacher educa- tors. A meta-analysis of inquiry teaching studies in science education by Sweitzer and Anderson (1983) reported that effective teacher preparation procedures included: (a) systematic observation of inquiry practices; (b) micro-teaching; and (c) feedback, in which supervisory conferences were combined with videotaped observations. More recently, Hutchinson (1989) found that pre-service teachers who participated in an inquiry- oriented seminar assumed more active teaching and learning roles than those teachers who participated in a traditional seminar set- ting. Fernandes (1988), who compared the ef- fects of explicit and implicit teaching of a Polya's (1957) heuristic model of mathemat- ical problem-solving, reported that both ap- proaches significantly enhanced the problem solving performance of pre-service teachers. However, only explicit instruction resulted in the conscious use of the heuristic. Fernandes concluded that in order to teach problem solving teachers must be competent problem solvers who are aware of the methods and processes that they employ. These studies support the idea that changes in ways of preparing teachers will result in changes in classroom performance. Moreover, as Wright (1990) stated, being a competent technological problem solver is, by itself, insufficient preparation to teach problem solving skills. Pedagogical skills and practices that foster students' problem solving abilities must be taught to prospec- tive teachers. PURPOSE OF THE STUDY Although a host of implications for in- struction have been offered from the research on problem solving in various domains, rela- tively few studies have addressed the need to prepare teachers to teach higher-order think- ing skills such as problem solving. Little is known about the experiences in which pre- service technology education teachers should participate in order to acquire the skills needed to be competent technological problem solvers and to use problem solving effec- tively as an instructional methodology in the secondary school classroom or laboratory. The purpose of this study was to develop a validated inventory of instructional proce- dures, techniques, and assessment methods that may be used by the profession as a framework for curriculum development and for the assessment of program effectiveness in the development of problem solving capabili- ties in pre-service technology teacher educa- tion programs. RESEARCH QUESTIONS Two sets of instructional practices were investigated: (a) procedures recommended to acquire the skills needed to be competent technological problem solvers, and (b) proce- dures that facilitate the use of problem solving teaching methods in the secondary school classroom or laboratory. Each of the two sets of instructional practices was or- ganized into three parts: (a) procedures re- commended to develop the problem solving capabilities, (b) instructional techniques for putting the procedures in place, and (c) methods for assessment of program effective- ness in delivering the procedures. Specif- ically, the following research questions were used to guide the study: 1. Are leading practitioners and advocates of problem solving instruction within the field of technology education in agree- ment with leading educators and psychol- ogists who are not in the field of technology education as to which proce- dures are effective in the development of problem solving capabilities? 2. What procedures are recommended to de- velop the technological problem solving capabilities of prospective teachers dur- ing pre-service technology teacher educa- tion programs? 3. What instructional techniques are appro- priate for the delivery of the procedures recommended to develop the technological problem solving capabilities of prospec- tive teachers? 4. How may the effectiveness of the proce- dures recommended to develop the techno- logical problem solving capabilities of prospective teachers be assessed? 5. What procedures should be included in pre-service technology teacher education programs to assist teachers in using a problem solving methodology? 6. What instructional techniques provide an effective means for delivering the proce- dures designed to assist teachers in us- ing a problem solving methodology? 7. How may the effectiveness of the proce- dures recommended to assist prospective teachers in using a problem solving meth- odology be assessed in pre-service tech- nology teacher education programs? PROCEDURES Perceptions of effective instruction were solicited from two selected panels of experts in problem solving. One panel was comprised of technology teacher educators (TECH) who were identified as leading practi- tioners or advocates of problem solving in- struction (n = 10). These panel members were identified from the a group of 22 leading technology teacher educators who previously served as Delphi panelists in the study by Householder and Boser (1991). The selection of TECH panelists was based upon their inter- est in problem solving as evidenced by (a) recent research, writing, and presentations on problem solving instruction, and (b) rat- ing of the importance of problem solving items in response to a questionnaire con- ducted by Householder and Boser. The second panel included leading educators and psychol- ogists (EXT) who have published in the area of problem solving and who were not in the field of technology education (n = 9). There were two reasons for using two panels. First, expertise from outside tech- nology teacher education may broaden the pool of instructional procedures recommended to the profession through this research. Barnes (1987), who indicated the need to broaden the curricular organizers of technology educa- tion, consulted practitioners in several pro- fessions outside of technology education. Second, utilizing two panels provided data for comparing the perceptions of the experts within technology education with the recomm- endations of experts external to the field. Potential panel members were contacted by telephone to seek their participation in the study and to establish a convenient time to conduct the telephone interview. Depend- ing on the amount of lead time prior to the scheduled interview, a letter confirming the scheduled interview time was either mailed or faxed to the panelist. Enclosed with the confirmation letter was a copy of the tele- phone interview schedule, a brief orientation to the study, and a listing of pertinent de- finitions. The interview times ranged from 10 minutes when the respondent had completed the survey in advance of the conversation, to 45 minutes when the items were reviewed and recorded during the discussion. In semi-structured telephone interviews, panelists were asked to rate the relevance of an inventory of procedures (70 items) synthe- sized from the literature, to comment on those procedures, and to suggest additional procedures that they considered essential in the development of problem solving capabili- ties. A 10-point scale was used by panelists to rate the procedures with a rating of 10 indicating that the procedure was absolutely essential. A rating of one implied that the recommendation was not relevant. The 10-point scale was assumed to have yielded interval data (Nunnally, 1978). This scale was selected because of the potential for in- creased reliability in comparison to scales with fewer intervals (Nunnally, 1978), and also because of its conversational appeal in an interview setting. That is, it is quite common for individuals to be asked to rate objects, ideas, or perceptions on a scale of 1 to 10. The telephone interview schedule was pi- lot tested with subjects not included in the research sample. The individuals who com- prised the sample for the pilot test were teacher educators who had recently completed doctoral research or who had a record of pub- lication in the area of problem solving in- struction. Descriptive statistics calculated for each of the 70 items included the combined mean score and standard deviation, the mean score and standard deviation by panel, the frequency of rating scores, and measures of kurtosis and skewness. Both the t-test pro- cedure and Wilcoxon's Rank-Sum Test were used to test for significant differences in the responses between the two panels. RESULTS AND DISCUSSION Of the 70 items rated by panelists, only the instructional methods of "Computer As- sisted Instruction" and "Lecture" received mean rating scores of less than 6 on the 10-point scale. Even these instructional techniques were the subject of mitigating comments from panelists as to their appropri- ate contexts in teaching problem solving. As a group, therefore, the procedures synthe- sized from the literature have a high degree of relevance for the preparation of pre- service technology education teachers. AGREEMENT BETWEEN PANELS Analysis of the results with both parametric and nonparametric statistical pro- cedures indicated that there was no signif- icant difference in the mean rating scores assigned to the items by the two panels. The overall mean rating score of the 70 items by the TECH panel was 8.07 on the 10-point scale. The mean rating score of the EXT panel was 7.76. The SD for both panels was 1.13. As a result of these findings, the combined mean scores of both panels (n = 19) were used to rank the inventory items. DEVELOPING TECHNOLOGICAL PROBLEM SOLVING CA- PABILITIES RECOMMENDED PROCEDURES. All 19 items in this section received a mean rating greater than 6 on the 10-point scale. Four proce- dures that emphasized practice in applying problem solving strategies in realistic con- texts and feedback on the use of those strat- egies received mean ratings of nine or greater. Other highly-rated items recom- mended that prospective teachers have the op- portunity to observe the regular modeling of problem solving behavior and the cognitive modeling of thinking processes involved in solving problems. The panelists' responses are reported in Table 1. TABLE 1 INSTRUCTIONAL PROCEDURES RECOMMENDED TO DE- VELOP TECHNOLOGICAL PROBLEM SOLVING CAPABILI- TIES --------------------------------------------- Mean SD Recommended Procedure --------------------------------------------- 9.42 0.69 Problem solving strategies are practiced in meaningful contexts 9.32 0.88 Feedback is provided on the use of problem solving strategies 9.24 1.15 Discussion questions empha- size "why and how" 9.05 0.91 Concepts and principles are connected to real world application 9.00 1.66 Problem solving behavior is regularly modeled 8.89 1.07 Alternative problem sol- utions are explored 8.83 1.04 Realistic problem situ- ations span the range of technological activities 8.63 1.26 Systematic verification processes are used to check results 8.63 1.50 Small group problem solving procedures are analyzed through inter- group discussion 8.42 1.57 Feedback helps teachers in- terpret their experiences 8.05 1.71 Problem solving thinking processes are regularly modeled through such practices as "talk aloud" methods and self-monitoring questions 8.00 1.83 Learning activities are linked to broad problem situations 8.00 1.87 Techniques and processes central to techno- logical activities are em- phasized through extended practice 7.89 2.66 General problem solving strategies (heuristics) are specifically taught 7.79 1.75 Sources of incorrect proce- dures are confronted 6.83 2.52 Worked-out examples are provided when appropriate 6.79 1.99 Concepts developed through problem solving activities are confirmed in discussion with more experienced persons 6.32 2.31 Prompts, such as check- lists, are readily avail- able to guide problem solv- ing performance 6.05 3.13 Initial learning of strate- gies focuses on the skill rather than content --------------------------------------------- TEACHING METHODS. Whereas there was considerable agreement between the panelists as to which procedures promote the develop- ment of problem solving abilities, no corre- sponding consensus developed on which instructional techniques might be used to fa- cilitate those procedures. With the excep- tion of small group problem solving experience, panelists' ratings of the tech- niques appeared to reflect familiarity with the practices. Members of the TECH panel tended to rate most highly those procedures practiced within the field, such as design- based problem solving, R & D experiences, and innovation activities. EXT panelists consid- ered techniques such as simulation and case study, which are perhaps more widely used in content areas outside of technology educa- tion, as appropriate delivery vehicles for the recommended problem solving procedures. Comments by several panelists emphasized the need to use a variety of instructional techniques. One panel member commented that all of the instructional techniques could be highly relevant in the proper context. More- over, as a panelist suggested, practice in applying problem solving skills in a variety of instructional settings may facilitate transfer of those skills to novel situations. Variety itself may have implications for the types of activities graduate technology edu- cation teachers chose to implement in their classrooms. Panelists ratings of the items in this section are reported in Table 2. TABLE 2 INSTRUCTIONAL TECHNIQUES THAT FACILITATE THE USE OF THE PROCEDURES RECOMMENDED TO DEVELOP TECHNOLOGICAL PROBLEM SOLVING CAPABILITIES --------------------------------------------- Mean SD Recommended Procedure --------------------------------------------- 9.29 1.10 Small group problem solving experience 8.61 1.42 Individual problem solving experiences 8.50 1.29 Simulation 8.44 1.92 Design-based problem solving 8.19 1.97 Cooperative learning 7.94 2.94 Research and development experience 7.89 2.35 Innovation activity 7.89 2.39 Invention activity 7.41 2.06 Community-based problem solving 7.25 2.29 Enterprise (class models a corporation) 6.71 2.73 Case study 6.53 2.39 Self-instruction through manuals etc. 6.50 2.17 Demonstration 6.44 2.30 Peer teaching 5.50 2.09 Computer assisted instruction (CAI) --------------------------------------------- ASSESSMENT OF PROGRAM EFFECTIVENESS. Of the eight assessment methods rated by the panelists, only "outcomes from group problem solving activities" had a standard deviation (SD) of < 1.00. For all the other items the SD was > 2.00. Although as a group the items are highly rated, the relatively large SD for these assessment methods suggested that there is little agreement among panelists as to the perceived relevance of these methods. Panel- ists' ratings of these items is presented in Table 3. The comments on the items run somewhat contrary to the item ratings. Panelists ex- pressed reservations about all but the three most highly ranked items. Several panelists were concerned about the "school smarts" of students. One panel member commented that structured interviews might not be a viable way to get at program effectiveness because "students know which answers are valued by the teacher." Panelists rankings and com- ments indicated a need for specific observa- ble measures from which to assess the effectiveness of problem solving capabili- ties. TABLE 3 METHODS FOR THE ASSESSMENT OF PROGRAM EFFEC- TIVENESS IN DELIVERING THE PROCEDURES RECOM- MENDED TO DEVELOP TECHNOLOGICAL PROBLEM SOLVING CAPABILITIES --------------------------------------------- Mean SD Recommended Assessment Method --------------------------------------------- 8.83 0.78 Outcomes from group problem solving activities 8.39 2.23 Performance samples of a specific problem solving phase 8.11 2.35 Examples of problem solving by the teacher 7.94 2.38 Written or verbal ration- ales for decisions 7.37 2.26 Structured interviews 7.00 2.27 Holistic scoring (points awarded for each stage of the problem solv- ing process) 6.95 2.06 Informal questioning during instructional activities 6.47 2.30 Teacher self-inventories of their problem solving abilities --------------------------------------------- TRAINING TEACHERS TO USE PROBLEM SOLVING TEACHING METHODS PROCEDURES THAT PROMOTE THE USE OF PROB- LEM SOLVING TEACHING METHODS. Ten of the 11 items in this section had a mean rating > 7.89. The limited range of the mean scores and the high mean ratings of the items indi- cate that the procedures have a high degree of relevance in assisting pre-service tech- nology education teachers in using a problem solving teaching methods. Visual categori- zation of the procedures suggests that the principal instructional component in promot- ing the use of the problem solving approach are: (a) practice with multiple forms of feedback, (b) opportunities to regularly ob- serve the modeling of problem solving in- struction and the associated cognitive processes, and (c) reflection upon the appli- cation of problem solving instruction in the classroom. The tabulated results of the items in this category are presented in Table 4. Panelists provided extensive comments on the items in this section and typically elab- orated upon an item or sought to combine ideas. For example, one panelist highlighted the importance of mediated observation TABLE 4 INSTRUCTIONAL PROCEDURES RECOMMENDED TO PRO- MOTE THE USE OF PROBLEM SOLVING TEACHING METHODS --------------------------------------------- Mean SD Recommended Procedure --------------------------------------------- 9.00 1.00 Problem solving theory is specifically linked to classroom practices of teachers. 9.00 1.32 Teachers receive multiple forms of feedback on their use of the problem solving approach (e.g. instructors, vide- otapes, and supervisory conferences) 9.00 1.49 Problem solving instruc- tional methods are regularly modeled 8.89 1.10 Teachers evaluate their own problem solving strategies and discuss their application to the teaching of children 8.63 1.64 Thinking processes used to facilitate problem solving instruction are regularly modeled through "talk aloud" strat- egies and self- monitoring questions 8.53 1.68 Teachers participate in the systematic observation of problem solving practices in the classroom and labo- ratory 8.42 1.61 Steps that comprise the problem solving approach are clearly de- fined and practiced in a microteaching environ- ment 8.26 2.05 Coaching in the use of problem solving methods is gradually re- duced as teacher competence increases 8.22 1.26 Lesson planning accounts for individual differences in students' problem solving abilities such as the con- fidence and competence of the problem solvers 7.89 2.13 Pre-service problem solving activities are similar to those that teachers will present to their technology educa- tion students 6.78 2.23 Teachers predict and visu- alize the outcomes of lesson planning --------------------------------------------- and multiple forms of feedback by stating that, "Any type of feedback can be useful, but it must be articulated feedback with spe- cific suggestions for improvement. Even in looking at a videotape, someone usually has to point out what to watch for." TEACHING METHODS. With the exception of lecture and case study, which had mean scores of 5.21 and 6.84 respectively, the mean scores of the other eight instructional tech- niques fell within a limited range from 7.53 to 9.11 on the 10-point scale. Student teaching was the highest rated technique in this group and the only item with a mean score > 9.00. Panelists' ratings of these items are reported in Table 5. Comments by panel members suggested that choice of technique is not necessarily as critical as factors related to the implemen- tation of the technique, such as frequency of use or appropriate sequencing during the teacher education program. While student teaching was the most highly ranked tech- nique, several panelists commented that this experience would only be useful if the coop- erating teachers were carefully selected. TABLE 5 INSTRUCTIONAL TECHNIQUES FOR THE IMPLEMENTA- TION OF THE INSTRUCTIONAL PROCEDURES RECOMMENDED TO PROMOTE THE USE OF PROBLEM SOLVING TEACHING METHODS --------------------------------------------- Mean SD Recommended Procedure --------------------------------------------- 9.11 1.37 Student teaching 8.72 1.41 Induction year 8.50 1.46 Simulation 8.39 1.61 Cooperative learning 8.16 1.46 Micro-teaching 8.00 1.56 Demonstration 7.79 1.58 Peer teaching 7.53 2.11 Discussion 6.84 3.07 Case study 5.21 2.59 Lecture --------------------------------------------- ASSESSMENT OF PROGRAM EFFECTIVENESS. Although there were only five items in this section, a visual examination of the mean re- sponses indicated two groups of assessment methods. The two most highly rated items, systematic observation of teacher performance during student teaching (9.00) and focused interviews (8.21), emphasized a structured approach to assessment. The remaining items relied on more indirect measures or self- report to assess teachers' use of problem solving teaching methods. The comments of panelists reflected a general skepticism of any form of assessment based on self-report by the learner. Addi- tionally, comments reinforced the need for assessment methods to be based on observed performance that can be checked against es- tablished benchmarks. The tabulated re- sponses to the items in this section are presented in Table 6. TABLE 6 METHODS FOR THE ASSESSMENT OF PROGRAM EFFEC- TIVENESS IN DELIVERING THE PROCEDURES RECOM- MENDED TO PROMOTE THE USE OF PROBLEM SOLVING METHODOLOGIES --------------------------------------------- Mean SD Recommended Procedure --------------------------------------------- 9.00 1.37 Systematic observation of teacher perform- ance during student teach- ing 8.21 1.68 Focused interviews 7.00 1.45 Children's performance dur- ing teachers' field experience (student teach- ing) 6.89 2.60 Journal reports from stu- dent teaching 6.79 2.37 Informal questioning during instructional activities --------------------------------------------- ADDITIONAL PROCEDURES SUGGESTED BY PANELISTS Panelists suggested 46 additional proce- dures which they considered essential for the development of the problem solving capabili- ties. Combining the individual suggestions resulted in a listing of 25 additional items. A pilot test to validate the relevance of these additional procedures was conducted as an adjunct to this study. On a mailed ques- tionnaire, panelists were asked to rate the additional procedures. The questionnaire format and item rating procedures used in the pilot test were identical to those processes used during the initial interviews. Eighteen of the panelists (94.7%) returned rating sheets. Analysis of the data indicated simi- larities in skewness, mean rating scores, and standard deviations between the additional procedures and the procedures recommended through the review of literature. Therefore, the additional procedures also appear to be relevant to development of problem solving capabilities. However, because of methodological differences only selected items will be discussed. TECHNOLOGICAL PROBLEM SOLVING. The most highly ranked instructional procedure, 9.11 on the 10-point scale, suggested that "Alter- native ways of looking at the problem should be considered in the search for a solution." Although this item appears to be a step in the technological problem solving process, it is also consistent with the information proc- essing concept of looking for a represen- tation of the problem that makes a solution more likely. The modeling of "looking for alternatives" as technical problems are ad- dressed may be a large step in promoting a problem solving thinking approach. In the assessment section, if "Instruc- tor models problem solving behavior" had been considered among the initial group of inven- tory items, it would have been the most highly rated item (mean rating = 8.89). Moreover, this was the only assessment method that did not relate program effectiveness in the delivery of problem solving instruction to some measure of learner outcome. Perhaps evidence of instructor modeling of problem solving behaviors is a powerful indicator of program effectiveness. TRAINING TEACHERS TO USE PROBLEM SOLVING TEACHING METHODS. The instructional proce- dures and techniques suggested in this sec- tion tended to elaborate on the more general items recommended in the initial inventory. Many of the items focused on some aspect of observational activity or field experience. However, the two most highly rated methods for the assessment of program effectiveness are clearly different from the items in the initial inventory. One suggested that, "Teachers analyze videotaped segments of their actual teaching or micro-teaching" (mean rating = 7.94), whereas the second item recommended that, "Teachers analyze situ- ations presented on videotape or videodisc in relation to specific program goals" (mean rating = 7.06). The use of these two methods may offer a controlled way to systematically examine program outcomes. SUMMARY AND CONCLUSIONS The inventory of procedures was highly rated by panelists and the increments between adjacent rankings were too small to establish meaningful cut-off points. Therefore no at- tempt was made to categorize the items within the sections. Clearly, in the correct con- text, all of the items may contribute synergically to the development of effective problem solving instruction. Given the small mean differences between adjacent rankings in the inventory, instructors and curriculum de- signers are advised to consider the proce- dures within each section as a group, and select procedures based on instructional ob- jectives and situational context. Further, as indicated by panelists comments, factors such as the frequency of use and appropriate- ness of the procedures or techniques at the learners' current stage of development must obviously be considered. In general, the ratings and comments by panelists indicated that the development of technological problem solving capabilities was typified by: (a) modeling and practice with feedback in realistic situations, (b) a variety of relevant instructional techniques, and (c) a collection of outcome measures to assess program effectiveness. Additionally, training teachers to use problem solving teaching methods involved: (a) modeling, me- diated observation, specific practice with feedback in using problem solving teaching methods, and reflective discussion on the ap- plication of these teaching methods; (b) carefully selected field experience sites; and (c) performance based assessment. Spe- cifically, the following conclusions were de- rived from this study: 1. The inventory of instructional proce- dures, techniques, and assessment methods compiled and rated through this research provide a relevant framework for the de- velopment of the problem solving capabil- ities in pre-service technology teacher education. 2. Procedures and methods advocated by tech- nology teacher educators were not signif- icantly different from those recommended by the EXT panel of authors and educa- tional psychologists. Therefore, it makes sense to utilize the expertise within the field of technology education when designing instruction intended to facilitate problem solving capabilities. 3. Instructional procedures that character- ized the development of technological problem solving capabilities included the: (a) application of problem solving strategies with appropriate feedback in variety of realistic situations, (b) ob- servation of behavioral and cognitive modeling, and (c) development of con- nections between concepts and applica- tions. 4. Although "small group problem solving" was the most highly ranked instructional technique for the development of techno- logical problem solving capabilities, panelist considered it appropriate and desirable to employ a variety of tech- niques. 5. Methods for the assessment of program ef- fectiveness in delivering technological problem solving instruction included: (a) outcomes from group and individual problem solving activities, (b) perform- ance samples, and (c) self-reports. 6. Instructional procedures that promote the use of problem solving teaching methods included: (a) the development of linkages between theory and practice, (b) multiple forms of feedback on practice teaching activities, (c) modeling of appropriate methods, and (d) mediated observation of problem solving instruction and reflec- tive discussion on the application of those teaching methods. 7. Field experience that is conducted in carefully selected sites and that empha- sizes problem solving teaching methods was identified as the most effective means of training pre-service teachers to use those teaching methods. 8. Systematic observation of teachers' per- formance during student teaching was the most highly rated method for the assess- ment of program effectiveness in promot- ing the use of problem solving teaching methods. IMPLICATIONS FOR TECHNOLOGY TEACHER EDUCATION Technology education teachers need to develop technical expertise, problem solving skills, and the ability to foster the problem solving skills of their students. These abilities will not likely occur by chance. The competencies needed to teach problem solving must be taught to prospective teach- ers. The inventory of procedures validated in this study may form a useful set of rec- ommendations for practice. These recommen- dations may serve to guide the selection of instructional practices, the development of curriculum, and the assessment of problem solving instruction in pre-service technology teacher education. Based on appropriate sections of the inventory, checklists may be developed to provide a formative assessment of the problem solving teaching methods used by instructors or to guide specific feedback to practice teachers. Research indicates that changes in ways of preparing teachers will result in changes in classroom perform- ance. The challenge to technology teacher educators is to select and implement the most effective teaching procedures. ---------------- Richard Boser is Assistant Professor, Depart- ment of Industrial Technology, Illinois State University, Normal, IL. REFERENCES Andre, T. (1986). Problem solving and educa- tion. In G. D. Phye and T. Andre (Eds.), COGNITIVE CLASSROOM LEARNING: UNDERSTAND- ING, THINKING, AND PROBLEM SOLVING (pp. 169-204). San Diego, CA: Academic Press. Barnes, J. L. (1987). An international study of curriculum organizers for the study of technology. DISSERTATION ABSTRACTS INTER- NATIONAL, 48, 05A, 1176. Barnes, J. L., Wiatt, C. & Bowen, M. A. (1990). The NASA/elementary technology ed- ucation project. THE TECHNOLOGY TEACHER, 49(4),13-17. Birch, W. (1986). Towards a model for problem-based learning. STUDIES IN HIGHER EDUCATION, 11(1), 73-82. Bransford, J. D., Goin, L. I., Hasselbring, T. S., Kinzer, C. K., Sherwood, R. D., & Williams, S. M. (1986). Learning with technology: theoretical and empirical per- spectives. PEABODY JOURNAL OF EDUCATION, 64(1), 5-26. Bransford, J., & Stein, B. (1984). THE IDEAL PROBLEM SOLVER. New York: Freeman. Carnevale, A. P., Gainer, L. J. & Meltzer, A. S. (1990). WORKPLACE BASICS: THE ESSEN- TIAL SKILLS EMPLOYERS WANT. San Francisco: Jossey-Bass. Commission on Pre-College Education in Math- ematics, Science and Technology. (1983). EDUCATING AMERICANS FOR THE 21ST CENTURY. Washington, DC: National Science Board. de Bono, E. (1990). The art of creative thinking. Teleconference presentation No- vember 15, 1990, PBS Adult Learning Ser- vices. DeVore, P., Horton, A., & Lawson, R. B. (1989). CREATIVITY, DESIGN AND TECHNOLOGY. Worcester, MA: Davis. Dewey, J. (1910). HOW WE THINK. Boston: Heath. Diaber, R. A. (1988). Discovery, inquiry and experimentation. In W. H. Kemp & A. E. Schwaller (Eds.), INSTRUCTIONAL STRATEGIES FOR TECHNOLOGY EDUCATION (pp. 166-182). Mission Hills, CA: Glencoe. Fernandes, D. M. B. (1988). Comparison of the effects of two models of instruction on the problem-solving performance of pre- service elementary school teachers and on their awareness of the problem-solving strategies they employ. DISSERTATION AB- STRACTS INTERNATIONAL, 50, 04A, 891. (Pub- lication No. AAC8913359). Gagne, R. M. (1985). THE CONDITIONS OF LEARN- ING AND THEORY OF INSTRUCTION (4th ed.). New York: Holt, Rinehart and Winston. Horath, L. D. (1990). Technological proficiences which should be developed during the preparation of teachers of technology education. Unpublished doctoral dissertation, Texas A&M University, Col- lege Station, TX. Householder, D. L., & Boser, R. A. (1991). Assessing the effectiveness of change to technology teacher education. JOURNAL OF TECHNOLOGY EDUCATION, 2(2), 16-31. Hutchinson, L. M. (1989). Conversational ana- lyses of inquiry-oriented and traditional teacher education seminars. DISSERTATION ABSTRACTS INTERNATIONAL, 51, 02A, 482. (Publication No. AAC9015877.) Hutchinson, P. (1987). Problem solving in British craft, design and technology edu- cation. DISSERTATION ABSTRACTS INTERNA- TIONAL, 48, 10A, 2607. (Publication No. AAC8720125). Kulm, G. (Ed.) (1990). ASSESSING HIGHER OR- DER THINKING IN MATHEMATICS. Washington, DC: American Association for the Advance- ment of Science. Lombard, A. S., Konicek, R. D., & Schultz, K. (1985). Description and evaluation of an inservice model for implementation of a learning cycle approach in secondary sci- ence classroom. SCIENCE EDUCATION, 69, 491-500. McCormick, R. (1990, October). The evolution of current practice in technology educa- tion. A paper presented at the NATO Ad- vanced Research Workshop: Integrating Advanced Technology into Technology Educa- tion, Eindhoven, The Netherlands. Nunnally, J. C. (1978). PSYCHOMETRIC THEORY (2nd ed.). New York: McGraw-Hill. Polya, G. (1957). HOW TO SOLVE IT. Princeton, NJ: Princeton. Savage, E., & Sterry, L. (1991). A CONCEP- TUAL FRAMEWORK FOR TECHNOLOGY EDUCATION. Reston, VA: International Technology Edu- cation Association. Sellwood, P. (1989). The role of problem solving in developing thinking skills. THE TECHNOLOGY TEACHER, 49(3), 3-10. Sweitzer, G. L., & Anderson, R. D. (1983). A meta-analysis of research on science teacher education practices associated with inquiry strategy. JOURNAL OF RE- SEARCH IN SCIENCE TEACHING, 20(5), 453-466. Thomas, R. G., & Englund, M. (1990). INSTRUC- TIONAL DESIGN FOR FACILITATING HIGHER OR- DER THINKING: VOLUME II: INSTRUCTIONAL DESIGN MODEL. St. Paul, MN: Minnesota Re- search and Development center for Voca- tional Education. Waetjen, W. B. (1989). TECHNOLOGICAL PROBLEM SOLVING: A PROPOSAL. Reston, VA: Interna- tional Technology Education Association. Wallas, G. (1926). THE ART OF THOUGHT. New York: Harcourt, Brace. United States Department of Labor. (1991). WHAT WORK REQUIRES OF SCHOOLS: A S.C.A.N.S. REPORT FOR AMERICA 2000. Washington, DC: The Secretary's Commission On Achieving Necessary Skills. Wright, T. (1990). Challenges facing educa- tors: The future of technology education. THE TECHNOLOGY TEACHER, 50(2), 3-5. Copyright 1993, Journal of Technology Education ISSN 1045-1064. Permission is given to copy any article or graphic provided credit is given and the copies are not intended for sale. Journal of Technology Education Volume 4, Number 2 Spring 1993