Innocent, 'Knowledge Based Simulation Models in Evolutionary Product Design: Communicating about Users', Arachnet Electronic Journal on Virtual Culture v3n02 (May 1, 1995) URL = http://hegel.lib.ncsu.edu/stacks/serials/aejvc/aejvc-v3n02-innocent-knowledge Electronic Journal on Virtual Culture ________________________________________________________________ ISSN 1081-3055 May 1, 1995 Volume 3 Number 2 EJVCV3N2 INNOCENT Knowledge Based Simulation Models in Evolutionary Product Design: Communicating about Users. Peter Innocent pri@dmu.ac.uk Abstract This paper considers and tests the proposition that knowledge based simulation models are useful in the development and testing of products. A case study is briefly described and generalisations are then presented within a rationale based on the principle of product evolution. Simulations are shown to be useful for supporting communication between individuals and groups about aspects of functionality and usability of the products. The result is a shared meaning relating to the product which is necessary for continuity in development. Finally, the proposition is made that future technological products will require knowledge based simulation models, not just for development but in order to dynamically enhance their usability and functionality. 1. Introduction. This paper starts with the proposition of Whiteside, Bennett and Holtzblatt (1988) that knowledge based simulation models are useful in the development and testing of products. They are assumed to be useful in that they catalyze communication between individuals and groups about aspects of functionality and usability of the products. The result should be a shared meaning relating to the product which is necessary for continuity in development. A related point of view is given in Rouse (1992) who has commented on the need for individuals to have mental models of the processes and other team members in order to perform effectively. A case study is briefly described which is directly based on the proposition that shared meaning is necessary. Generalisations are then made from the case study and presented within a rationale based on the principle of evolution. Finally, the proposition is made that future technological products will require knowledge based simulation models, not just for development but in order to dynamically enhance their usability and functionality. 2. The Case Study: Telephone interfaces. Among everyday objects, the telephone is unique in that it is an enabling platform for many other communication tasks. Although we think of a telephone as an everyday object, it was only patented in 1876 and the first commercial network came into service in 1878 with 20 phones. By the mid 1980Us, there were predicted to be around 30 million phones in the UK alone (Rutter (1987)) . The vast expansion of the telephone network has revolutionised communications and is continuing to do so with, for example, portable phones, video phones and integration with home information systems through cabled networks. From its humble beginnings as a simple voice operated device connecting two people via an operator and 2 wires, the modern telephone now communicates with a very large distributed network of mainly digital exchanges. Computing technology now provides many features such as conferencing, call back, camp-on busy and so on. and home phones may have access to these features. Competition is forcing telephone companies to devise ever increasing numbers of additional and soometimes inter-related features. Recently, there has been a curious mix of the old and new technologies in the eastern block countries. Users have been known to use a phone with no dial connected to a modern exchange and select features by tapping the hook! However, most telephone companies are concerned with ensuring that the interfaces to the phone system are matched to the available functionality required by the user. Norman (1988) has many references to the phone interface as a source of endless problems in using features - even basic ones such as making a call. He describes telephone evolution as a process wherein a feature tends to continue to exist in new product generations if negativity is not associated with it. This is partly because telephone interfaces are tested for usability using a variety of methods (e.g. Maskery (1984) and Kennedy (1989)) and partly because of upwards compatability requirements in technology products. However, good and well tested features from the users' point of view (such as providing tactile and auditory feedback) can be lost due to technological advances which make the phone smaller with silent parts. In recent years, technological advances associated with miniaturisation and compression have accelerated to the extent that modern phones enable many features on a small interface, some of which are dependant on others. Principles of design are enunciated by Norman, such as good mapping between the users action and the product response, providing feedback and making things visible. These can be difficult to apply in the context of rapid technological change since functions of an early generation tend to be subsumed by functions of a later generation. As Norman (op cit.) states, the multiple forces of a competitive market means that new models of a product do not necessarily benefit from the experience of the previous model. He cites a case concerning the physical ergonomics of the phone. Negative forces include the demands of time (new models are into design before old ones have been released to customers) and the need for individuality (the signature of a company and designer). Telephone companies place large constraints on the testing which can be done in these circumstances. The application of these constraints can lead to a situation where testing is done rapidly and too late in the design process to make any significant difference to the usability of the product. Usability test reports tend to be made as quickly as possible on specific products and have little generic use to other related products used by similar users performing similar tasks. Development engineers can offer considerable resistance to change suggested by usability testers particularly when late in the development process and must be convinced of the need for it in a language they understand (Kennedy 1989). That is, they must share meaning about the product and this is difficult to achieve. The situation is worsened when, inevitably, the developers and testers of these products move on into careers and long term expertise in shared meaning is lost. In these circumstances, a strategy is needed for countering the negative effects of the forces of the competitive market on the loss of usability of the product. A strategy that has been developed in house by a large North American telephone organisation is to simulate the functionality of the new product as early as possible in the development stage. This then brings forward the stage at which a prototype can be mad available for usability testing. However, this does not address the problems of the lack continuity in the shared meaning relating to products over many generations of testers and products. There are a number of possible strategies (non-exclusive) which may be adopted. For example, the adoption of standards and suitable training for the usability testers. The strategy discussed here is to use a simulation of a "standard user" to test the simulated phone with in the early stages of development. The simulation of the standard user is then an embodiment of the shared meaning between the groups at any time. 2.1 Simulation of a Phone. The user interface and actions of a phone (not the hardware interface) using object oriented tools such as Hypercard (TM) on a Macintosh (TM) (Innocent (1989)) have been simulated using a suitable specification. It is possible by using touch sensitive screens and enhanced image generation, to increase the realism of the simulation for usability testing by real users. But the major advantages of developing simulations like this are reusability of code for the different product and tester generations and proving a design early on in the development process. To be successful, engineers must provide clear specifications of the functions of the phone and it's relationship to the network early in the design process. The development of the simulation then proceeds in parallel with the development of the actual system - each feeding off the other through a common operational commitment. This process is an aid to rapid prototyping (Wilson and Rosenberg (1988)). It is particularly valuable for interface design as icon design and layouts are a time consuming development issue. Simulation builders must thoroughly understand the phone operation as well as be reasonably expert in building simulations and testing with them. Early specifications can be incomplete and incorrect and this can be difficult to determine from formal specification documents because the meaning of the document is not clear to usability testers. Despite the positive aspect of a simulation as a shared meaning between developer and usability tester, usability testing with a simulated phone may lead to results which are not valid or generalisable to real users of a real phone. This is an acceptable risk since the simulated phone is available for testing by users or by other methods such as the application heuristics by knowledgeable testers (Nielson and Molich 1991). The validity of the early test procedures can be tested after further testing with the real product after development. Suitable changes can then be made. 2.2 Simulation of Phone Users. The process of developing a simulation of users requires that there is an agreed reference model (a "User Reference Model" or URM) which acts like a design specification. The URM contains both declarative knowledge (what the user knows) and procedural knowledge (what the user has learned to do). At the time of development of the initial URM, there was a serious lack of consensus in the testing group about the form and contents of a URM. These arise from many sources: the differing backgrounds of the people and their training as well as their levels of experience. The group also suffered from too little time for discussion to resolve these differences and so agree on a shared meaning. The development of a URM was supported with enthusiasm within this context as it provided a means for sharing meaning concerning the focus of their work (Whiteside, Bennett and Holtzblatt 1988). This was clearly one of the prime reasons for pursuing the work as they were working in relative isolation on different or remotely related products. The URM is intended to evolve with use but the starting point is clearly of importance as it is a framework for further development. Given the lack of consensus, a minimalist approach was taken and the least contentious and most generic structure was adopted. Complex interfaces forces testers to use models of users which include higher cognitive functions. Thus the initial URM is based upon a simple taxonomy of users and tasks (Rich 1983), a simple generic perception/action model of how people use everyday objects (task action generators and scripts) and a simple model of memory and problem solving (a production system). Within these components, the ideas of mental models (Norman (1983)) and belief systems have been incorporated. The basic URM may be thought of as a definition of the space of possibilities for evolutionary models of the users performing tasks with phones. It is a very simple model compared to, for example, those described by Kobsa and Whalster (1989) which are in the HCI context and intended to be generic (e.g. Kelly (1988)). The initial model of phone users borrows components from AI (e.g. production systems) and HCI (e.g. task action grammars (TAG), Payne and Green (1986)) and attempts to integrate them so that the operation of the complete model can support an interpretation/action stream within its context of use unless or until there is a breakdown (Winograd and Flores 1985). At this point, some simple recovery plans come into operation which involve backtracking and synthesis of old task scripts into new ones. The initial URM does not concern itself with learning or complex error recovery although provision for explanation generation has been made. Neither does it include other possibly important components such as the users perception of an interface and the system of labelling icons (if used on an interface). Whether or not these will be developed depends on the usability testing group. However, it is expected that these aspects will evolve rapidly since breakdowns are a principal focus of attention for usability testers. Once a URM was established, it was possible to produce a working simulation of a phone user and show how it would interact with a simulation of a phone. A working simulation was produced by writing a URM interpreter in PROLOG (the simulated URM or SURM -Innocent (1989)). This takes the type of phone, the users' task and the characteristics of the user as an input and generates sequences of simulated user actions as output. The simulated actions are presented to the simulation of the phone automatically which then acts on them and changes state. The SURM then generates another action and the cycle continues until the task is completed or the task fails to complete. A side effect in the process is the generation of a log containing information about how the SURM was working internally. For example, the contents of the simulated users short term and long term memory are in the log after every action/response. A post processor of this log is necessary in order for automatic reporting of the experiment to be achieved. This has yet to be written. However, the SURM combined with the simulated phone system provides very early focusing on particular issues relating to the usability of the product. It is a tool which has both short term and long term advantages. These are, for example, identifying what experiments should be done and for what purpose and providing a basis for shared meaning within and between interested groups. Developments of the simulation tool included the provision of a simple interface between the simulation of the user and the simulation of the phone. This was identified as an important issue along with automatic reporting of results of testing. 2.3 Case Study Conclusions. The success of the approach depends on the adoption by the organisation of implicit standards which are embodied in the simulations. This is not a trivial issue as standards can be seen as possible limitations on design and hence loss of competitive edge. This must be traded off against other factors such as the preservation of good design through the different generations of product. There is a cost involved with the development of simulations although this is relatively minor compared with the cost of not developing them. There is a large hidden cost in the lack of social cohesion which can develop in the interacting parties. This results in misunderstandings, time loss, financial penalties and poor morale. 3. Generalisation Issues. While the case study has identified particular issues in particular circumstances, which support the view that simulations are important to enable shared meaning, it is important to identify the limits within which this view can be generally valid. 3. 1 Are phones representative of classes of products? There are clear differences in usability requirements between household appliances (and other everyday objects) and specialist products to be used in a particular context. In the latter case it is reasonable to expect that users receive training. However, it is becoming increasingly expected that even specialist products should be Tuser friendlyU and require little, if any, expensive specialist training. This is largely because the user interface to the products are required to make task accomplishment Tself evidentU in some way, perhaps by mimicry of an everyday object. The use of computer technology for user interfaces to all products (specialist and everyday use) is becoming increasingly common and de facto standards and styles have appeared for their use. The use of the Twhat you see is what you getU metaphor is intended to make the accomplishment of tasks self evident. However, even for everyday objects, simple mappings of existing electromechanical operator equipment into a symbolic form for a computer interface may not be so easily accomplished. Many consumers prefer that the new systems and appliances are at least as usable as the old ones they are replacing, but this can be difficult to achieve because of higher level merges of functionality. This forces users to perform TcognitiveU tasks rather than simply following a previously learned chain of actions relating to the state of the appliance to achieve a task. An example of such a cognitive task would be to work out how to get the appliance in the right state so that a given function can then be used. In computing, there is a trend towards integrating applications such as spreadsheets, data bases and word processing thus making cognitive tasks even more complex for the user. Recently, this trend has been taken further to avoid the problems associated with building increasingly large applications. Computer scientists have developed the idea of providing dynamically linked multiple micro-applications supported in a suitable environment. Thus, when an upgrade is made to, say, an editor within a DTP package, it will only mean a small change for the user's existing software and not a major release of a new version of the product. However, from the user's point of view, such an advance, though welcome for economic reasons, may well mean an increase in confusion when there is a lack of compatibility between products which appear to be the same package. As user's differ in their take up of options to upgrade within a package, there will be increasing diversity of functionality between them. There is thus a technological convergence of all products towards multifunctional devices with sophisticated interfaces which should be self-evident in use. The phone is an example par excellence of incremental function design which must conform to constraints relating to previous generation of the product and users. The phone is simply a particular instance of a large general class of products. Therefore, the simulation approach taken here could be used in any similar product development context such as radios, televisions and washing machines and most common electrically based household appliances and systems. 3.2 How useful is the model of the User? A general concept of how people live in the world has been presented by Winograd and Flores (1985). They develop the notion that existence in the world means continuous interpretation and action with it and that this leads to a new design paradigm for computer based systems. In this paradigm, the aim is to achieve products which are Tready-to-handU and do not TthrowU the user or lead users to Tbreak downU the world into objects and properties. Winograd and Flores (op cit.) present a view of cognition as a biological phenomenon. When trying to model users, they suggest behaviour can be described in the cognitive domain (where purposes and couplings are important) and/or as a structure determined system (where actual reflex paths are the key). These models lead to an understanding of how users may interact with products which are part of their world and this is the subject of an entertaining book by Norman (1988). He considers the psychology of everyday things (POET) such as car controls, telephones and tea machines. Norman shows that even with simple products it is very easy to design and build them so that they are difficult to use without making errors. Principles of design are enunciated such as mapping rules (user actions to device responses), visibility (of the states of the device) and feedback (responses to actions). The model of the user he proposes is built on the idea that users develop mental models of the world they are interacting with. This idea is taken from an earlier paper (Norman 1983). Mental models are used for a variety of reasons: predictions of the devices responses; explanations of why the device reacted the way it did and so on. Although a simple idea, there are many varieties of interpretation of it. This is because a mental model is a complex concept which embodies the userUs procedural and declarative knowledge of the world and has many levels. For a further related discussion of these ideas see Paul and Thomas (1994). Rouse (1992) has commented on the need for individuals to have mental models of the processes and other team members in order to perform effectively. The URM can be seen as an embodiment of a groups mental model of particular aspects of context. Without such a model it is clear that large differences would be expected between group members (Gillan, Breedin, Cooke 1992). It is clear that artificial intelligence is a main approach to the computational aspects of simulation of users (Spiegel and LaVallee 1988). The general strategy of combining artificial intelligence techniques with simulation is progressing in many areas (Pidd 1989, OUKeefe and Roach 1987). Specialists in human computer interaction (HCI) have been aware of these ideas for some time and have attempted to model users at many levels in order to predict errors (break downs) and the times to perform a task and to explain the userUs performance (Rasmussen 1990). Human factors HCI specialists have developed a TGoals, Operators, Methods and Selection (GOMS)U model which allows inferences to be made about users performance in a particular context (Card, Moran and Newell 1983). Other models are task action grammars (TAG), (Payne and Green 1986) and the command language grammar (CLG). (Moran 1981). Most of these models are based around using computer systems for simple tasks such as editing text or using mail boxes. They are context sensitive in that one model derived in one context cannot usually be directly used in another context without modification. The models may not be appropriate to products which have a computer interface but have limited functionality. As Green (1992) points out, the problem with modelling people who are performing cognitive tasks with a product is that we are drawn into extreme complexity very quickly. It can lead to the position where the whole of cognitive science must be solved before we can proceed. In this case we would have an unlimited theory of how people act and all we need to do is provide contextual data. Indeed, artificial intelligence has been attempting to provide some unlimited computational models for some time with limited success. These cover, for example, natural language processing, planning, problem solving and game playing. Fortunately, in the limited context of usability testing of particular products, we can adopt a pragmatic position where the use of limited theories could be sufficient. Limited theories are embodied in, for example, GOMS, TAG and CLG. The difficulty is that the tester needs to know how to use and combine these in a particular context In the normal iterative development cycle, this knowledge must be acquired by experience over long periods of testing. The minimalist approach taken with the phone system has shown that integration of disparate methods is not trivial and is subject to severe criticism on a theoretical basis. However, the resulting model, though possibly incomplete, is reasonably context free and in the simulation, the knowledge bases and processing are distinct and could be easily adapted. The problem is then more knowledge acquisition for use in another context. Holland has been quoted in Mitchell-Waldrop (1993, p147) as stating that "to get a deep understanding of complex adaptive systems we need maths, simulation techniques that analyse internal models, the emergence of new blocks and the rich levels of interaction between multiple agents." There is no more complex adaptive system than human beings. This work supports this statement and it is hoped that simulation approaches may help to reduce the number and variety of models being proposed in the various areas of research by providing a shared meaning between researchers in modelling people. 3.3 Is the model of the context in which the product is developed representative? The basic model being presented of phone development is one of iterative design and development in a competitive environment. i.e. an evolutionary model. Simply stated, evolution is about the survival of the fittest in a changing environment. Consequently, the fittest is not necessarily the most functionally adequate in a global sense but simply that which is adequate at a particular time when the environment suits it. However, in a complex adaptive system such as those involving people and technology, Mitchell-Waldrop (1993) describes the process as one of continual unfolding where 'agents' exploit niches which leads to other niches and so on. In this model of evolution there is a continuous state of inequilibrium. Agents can never fully optimise their fitness although improvements are possible relative to each other. Using this model of evolution, if an environment changes slowly in relation to the possibility of changes made by an agent to a product, then it may be possible to optimise the product with respect to that environment. In this case we can have an Tecological ergonomicsU paradigm (Carroll 1989) which optimises the products usability within its lifetime (given all other factors are equal - e.g. cost, functionality, availability) and so becomes the fittest and survives in the market place. Iterative evaluation has been shown to be a suitable methodology for achieving this optimisation (Hewitt 1986). The conditions for ecological ergonomics are severe. Some modern products are new innovations - creationist competitors which will sink optimised dinosaurs without trace. However, for some classes of products, the conditions can be met for the most part. In particular, everyday objects such as the radio can undergo major technological revision but from the usersU point of view present the same functionality. In this sense there is Tparental respectU (Radcliffe 1992). In the Winograd and Flores (op cit.) point of view, the user is looking for certain actions to produce specific responses of the product in order to maintain the flow of interpretation and action. The means for carrying out those actions may differ and thus can be optimised in relation to this flow. The case study shows how brittle the context is for achieving ecological ergonomic success. Ergonomics is in competition with product development for time and resources, yet the overall fitness of the product is better tested in house first rather than in the market place. Hence there should be a niche for ergonomic testing to take place. 4. The future. Technological advances mean that we no longer need to consider the interface for advanced products to be hard wired for the user and optimised by outside agents. It is a very small step from existing products to make them customisable by the users and this is already possible with many home systems. Other levels of adaptivity within products are possible (Totterdal et al 1992) and many require techniques of maintaining models of the users (user models) in order to adapt. Typically, such a system will keep a record of user's actions with a system and provide 'short cuts' for the user by taking actions on behalf of the user. These user models are likely to be very similar to those which have been developed in the case study in this paper. The ultimate user interface is self adapting to the user - a natural language interface in the true sense. A knowledge based adaptive system (KBAS) like this is unattainable at present (and possibly not ever) except in a highly constrained environment although many have been proposed (Norcio 1989, Innocent 1982). The search to find good reliable and efficient methods for such an interface is proving difficult. Research is currently going on into how such systems could be built (Benyon, Innocent and Murray 1987, Benyon, Murray and Milan 1987). There are many difficulties with a KBAS system other than the obvious one of building it. A self adaptive interface has a degree of control over the user and this may be unacceptable to certain groups of users. 5. Conclusion. This paper has presented a view of the use of modelling and simulation in the context of an iterative design paradigm. A particular case has been described in outline and generalisation of the issues has been discussed. It is concluded that the case supports the view of Whiteside, Bennett and Holtzblatt (1988) that knowledge based simulation models are useful in the development and testing of products in that they catalyze communication between individuals and groups about aspects of functionality and usability of the products. The result is a shared meaning relating to the product which is necessary for continuity in development. After consideration of some of the issues relating to generalisation it was concluded that simulations would become more necessary as a result of technological development and the pace of change. In this context, they also act as counter forces to negative competitive forces which can result in the premature release of a product. Finally, it is proposed that developments of simulations of users will form a major subject of development for future generations of technology which will incorporate self adaptive interfaces and functionality. References and Bibliography: Ackermann D. and Greutmann T. (1987). Experimental reconstruction and simulation of Mental models. In RMental Models and HCI 1S edited by D. Ackermann and M.J. Tauber. pp133-150. Elsevier Science Publishers. Aufderheide P. (1987). Universal Service: Telephone Policy in the Public Interest. Journal of Communication, 37, pp81-85. Benyon D. and Murray D. (1988). Experience with Adaptive systems. The Computer Journal, 5, pp465-473. Benyon D, Innocent P.R. and Murray D (1987). System Adaptivity and the Modelling of Stereotypes. Proceedings of 2nd IFIP Conference on HCI. INTERACT 1987. H.J. Bullinger and Shackel B. (eds). pp245-253. Elsevier Science. Benyon D., Murray D. and Milan S. (1987). Modelling Users' Cognitive Abilities in an Adaptive System. Proceedings of the RISO (Danish Institute for research in informatics) conference on Cognitive Ergonomics, December 1987. Cacciabue P.C. and Bersini U. (1988). Modelling Human Behaviour in the Context of a Simulation of Man-Machine Systems. In: Training, Human Decision making and Control, edited by J. Patrick and K.D. Duncan. Noth- Holland., pp223-241. Card S. , Moran T. and Newell A. (1983). The Psychology of Human- Computer Interaction. LEA press, ISBN 0-89859-243-7 Carroll J. M. (1988). Mental Models in Human Computer Interaction. Chapter 2 in the THandbook of HCIU, M.Helander (ed), Elsevier Science, Pub. Carroll J. M. (1989). Evaluation, Description and Invention: Paradigms for HCI. In Advances in Computers, Vol 29, pp47-77. Cashdan A. and Jordin D. (1985). Studies in Communication. Blackwell, Oxford. De Greene K.B. (1991). Emergent Complexity and person-machine systems. Int. J. Man-Machine Studies, 35, pp219-234. Finin T.W. (1989). GUMS - A General Modelling Shell. In Kobsa A and Whalster W (eds). User Modelling in Dialog Systems, Springer-Verlag. Gillan D.J., Breedin S.D., Cooke N.J. (1992). Network and multidimensional representations of the declarative knowledge on human-computer interface design experts. Int. J. Man-Machine Studies, 36, pp587-615. Gould J.D. (1988). How to Design Usable Systems. Chapter 35 in the THandbook of HCIU, Heilander M. (ed), North-Holland. Green T.R.G. (1992). Limited Theories as a framework for HCI. In RMental Models and HCI 1S edited by D. Ackermann and M.J. Tauber. pp3-40. Elsevier Science Publishers. ISBN 0 444 88453 X Hewett T.T. (1986). The role of iterative evaluation in designing systems for usability. p196 in Proceedings of the 2nd Conf. of the BCS HCI specialist group, RPeople and Computers: Designing for UsabilityS edited by Harrison MD and Monk AF. Mitchell-Waldrop M. (1993). Complexity. Published by Holland, ISBN 0- 670-85045-4 Innocent P.R. (1989) Usability testing: A practical approach to Mental Model Construction. Proceedings of 8th Interdisciplinary Workshop on "Informatics and Psychology: Mental Models and Human_Computer Interaction", Scharding, Austria. ISBN 0 444 88602 8 Innocent P.R. (1988). The Many Faces of Human-Computer Interaction. Computer Bulletin, Vol. 4., Part 3, pp33-36. Innocent P.R., van der Veer G.C., Waern Y., Tauber M. (1988). Representations of the User Interface. pp 345-359 in "Research into Networks and Distributed Applications". R. Speth (ed). Elsevier Science. Innocent P.R., Van der Veer G.C. and Waern Y. (1987). Experiences with UNIX mail: An Experiment. EUUG Conference proceedings, Trinity College, Dublin. Innocent P.R. (1982). Towards self adaptive systems. International Journal of Man-Machine Studies, Vol 16., pp287-299. Jeffries R., Miller J.R., Wharton C., Uyeda K.M. (1990). User Interface Evaluation in the Real World: A Comparison of four techniques. Proc. ACM CHI 90, pp235-241. Kelly C., (1988). User Modelling and User Interface Design. In TPeople and Computers VII, BCS Publication, Cambridge University Press. Kennedy S. (1989). Using Video in the BNR Usability Laboratory. SIGCHI Bulletin, 21,2, pp92-95. Kobsa A. and Whalster W. (Editors) (1989). User Modelling in Dialog Systems. Springer-Verlag. Maskery H.S. (1984). Adaptive Interfaces for Naive Users. Interact 1984 Proceedings of IFIP Conference. pp343-350. Moran T.P. (1981). The Command Language Grammar. IJMMS, 15, pp3-50. Neilson J. (1992). Finding Usability Problems through Heuristic Evaluation. Proc. ACM, CHI 92, pp373-380. Neilson J. , Molich R. (1992). Heuristic Evaluation of User Interfaces. Proc ACM, CHI 90, pp249-256. Norcio A. F. (1989). Adaptive HCI: A literature survey and perspective. IEEE Trans. on Systems, Man and Cybernetics. 19, 2, pp399-408. Norman D.A. (1983). Some Observations on mental models. In: A.L. Stevens and D. Gentner (Editors), RMental ModelsS, Erlbaum, Hillsdale N.J. Norman D.A. (1988). The Psychology of Everyday Things. Basic Books. OUKeefe R.M. and Roach J.W. (1987). AI Approaches to Simulation. J. of O.P.Soc., 38, pp713-722. Paul R.J. (1991). Recent developments in Simulation Modeling. J. Opl. Res. Soc., Vol 42, No 3., pp217-226. Paul R.J. (1989). Combining AI and Simulation. In RComputer Modeling for Discrete SimulationS , M.Pidd (editor), Wiley. Paul R.J. (1989). AI and Simulation Modelling. In RComputer Modeling for Discrete SimulationS , M.Pidd (editor), Wiley. Paul R.J. ad Thomas P.J (1994). Computer Based Simulation Models for problem solving: Communicating problem understandings. The Arachnet Electronic journal on Virtual Culture, ISSN 1068-5723. Vol 2, No 2. Payne S. and Green T. (1986). Task Action Grammars - a model of the mental representation of task languages. Human-Computer Interaction, 2, pp93-99. Pidd M. (1989). (Editor). Computer Modelling for Discrete Simulation. John Wiley and Sons. Radcliffe N.J. (1992). Forma Analysis and Random Respectful Recombination. Proc. of the 4th International Conference on Genetic Algorithms. Rasmussen J. (1990). Mental Models and the Control of Action in Complex Environments. In RMental Models and HCI 1S edited by D. Ackermann and M.J. Tauber. pp41-69. Elsevier Science Publishers. ISBN 0 444 88453 X Rich E. (1983). Users are Individuals: Individualising User Models. Int. J of Man Machine Studies, 18, pp199-214. Rouse W.B. (1992). The Role of Mental Models in Team Performance in Complex systems. IEEE Trans on Sys, Man and Cyb, Vol 22., No 6., pp1294- 1308. Rozenblit J.W., Hu j., Kim T.G., Zeigler B.P. (1990). Knowledge based design and simulation environment: foundational concepts and implementation. J. of O.R., Vol. 41, No 3, pp475-489. Rutherford A. and Wilson J.R. (1989). Models of Mental Models: an ergonomist- psychologist dialogue. In RMental Models and HCI 2S edited by M.J. Tauber. and D. Ackermann, pp39-55. Elsevier Science Publishers. Rutter D. (1987). Communicating by Telephone, Pergammon Press, U.K. Spencer R.H. (1989). Computer Usability Testing and Evaluation. Prentice- Hall Inc., pp104-105. Spiegel and LaVallee (1988). Using an Expert System to drive a simulation experiment. AI papers 1988, pp108-112. ISBN 0-911810-35- 9. Totterdell P.A., Norman M.A., and Browne D.P. (1987). Levels of Adaptivity in Interface Design. In Proceedings INTERACT 87, 2nd. IFIP Conference on HCI (Editors: Bullnger and Shackel B.). Tyler S.W. (1989). An Interface architecture to provide adaptive task- specific context for the user. Int. J. Man-Machine Studies 30, pp303-327. Uttamsingh R.J. (Editor) (1, 988). AI Papers. The Simulation Series, Vol 20, part 1. The Society for Simulation, San Diego. Van der Veer G.C., Tauber M., Waern Y., and van Muylwujk B. (1985). On the interaction between system and user charactersitics. BIT, 4(4). Waern Y. (1987). Mental Models in Learning Computerised Tasks. In RPsychological Issues of HCI in the WorkplaceS, Frese M., Ulich E and Dzidz W (editors), Elsevier Science. Welch J. (1981). Electronic Mail Systems - A practical evaluation Guide. Computer Centre Press, England. Whiteside J., Bennett J., Holtzblatt K. (1987). Usability Engineering: Our experience and evolution. In RHeilander M., The Handbook of HCIS, North- Holland. Wilson J and Rosenberg D (1988). Rapid Prototyping for UID. Chapter 13 in the THandbook of HCIU, Heilander M. (ed), North-Holland. Winograd T and Flores F (1985). Understanding Computers and Cognition: A New Foundation for Design. ABLEX Publishing Company Co., Norwood, N.J. _________________________________ Articles and Sections of this issue of the _Electronic Journal on Virtual Culture_ may be retrieved via gopher to refmac.kent.edu 70 or via anonymous ftp to byrd.mu.wvnet.edu or via e-mail message addressed to LISTSERV@KENTVM.KENT.EDU (instructions below) http://www.marshall.edu/~stepp/vri/ejvc/ejvc.html or GOPHER refmac.kent.edu Papers may be submitted at anytime by email or send/file to: Diane K. Kovacs, Editor-in-Chief, _Electronic Journal on Virtual Culture_ ejvcedit@kentvm.kent.edu _________________________________ *Copyright Declaration* Copyright of articles published by Electronic Journal on Virtual Culture is held by the author of a given article. If an article is re-published elsewhere it must include a statement that it was originally published by Electronic Journal on Virtual Culture. The EJVC Editors reserve the right to maintain permanent archival copies of all submissions and to provide print copies to appropriate indexing services for for indexing and microforming. _________________________________ _THE ELECTRONIC JOURNAL ON VIRTUAL CULTURE_ ISSN 1068-5327 ____________________________ WWW instructions ____________________________ http://www.marshall.edu/~stepp/vri/ejvc/ejvc.html ____________________________ GOPHER Instructions ____________________________ GOPHER to refmac.kent.edu 70 /Electronic Journals ____________________________ Anonymous FTP Instructions ____________________________ ftp byrd.mu.wvnet.edu login anonymous password: users' electronic address cd /pub/ejvc type EJVC.INDEX.FTP get filename (where filename = exact name of file in INDEX) quit LISTSERV Retrieval Instructions _______________________________ Send e-mail addressed to LISTSERV@KENTVM (Bitnet) or LISTSERV@KENTVM.KENT.EDU Leave the subject line empty. The message must read: GET EJVCV2N2 CONTENTS Use this file to identify particular articles or sections then send e-mail to LISTSERV@KENTVM or LISTSERV@KENTVM.KENT.EDU with the command: GET where is EJVCV#N# (Volume and Issue Number of EJVC) and is the first Author's Lastname.