Seminaarin alkusivu

Käytettävyystutkimuksen menetelmät -seminaari

Kevät 2004

Ajankohtaista | Työmuodot | Lukemisto | Aikataulu | Palautelomake

Seminaarin lukemisto

Lyhenteellä ACM DOI merkityt artikkelit ovat saatavilla ACM:n digitaalisesta kirjastosta, ja pelkkä DOI viittaa Science Direct -kirjastoon. Näihin on pääsy Tampereen yliopiston verkossa olevilta tietokoneilta. Hakasulkeissa olevat lähteet [PDF*] on koottu seminaarilaisten käyttöön hakemistoon, jonne vaaditut tunnukset kerrotaan seminaarissa.

Tässä annettu aiheiden lista muotoutuu seminaaritöiksi osanottajien kiinnostuksen mukaan. Tarkoitus on, että jokainen perehtyy oman aiheensa lisäksi kahteen muuhun aiheeseen, joita seminaarissa opponoi. Useimpiin aiheisiin löytyy paljon muutakin aineistoa, jota kaikkea ei ole tässä esitelty. Seminaarityöhön pitää hakea itse lisää lähteitä. Yksi hyvä lähtökohta on yhteisten artikkelien lista.

suositeltu lähde, [PDF*] vaatii tunnuksen

Aihepiiri Lähdeaineistoa
Kyselylomakkeet ja haastattelu
Kyselylomakkeet (Questionnaires)
  • Andrews, D., Nonnecke, B. & Preece, J. (2003) Electronic survey methodology: A case study in reaching hard-to-involve Internet users. International Journal of Human-Computer Interaction, 16 (2), 185-210. draft versions available: PDF tai PDF
  • Hufnagel, E.M. & Conca, C. (1994) User response data: The potential for errors and biases. Information Systems Research, 5:1, 48-73.
  • Ives, B., Olson, M.H. & Baroudi J.J. (1983) The measurement of user information satisfaction. Communications of the ACM 26 (10). ACM DOI
  • Kirakowski, J. (1994) The use of questionnaire methods for usability assessment. Background notes on the SUMI questionnaire. HTML
  • Nielsen, J. (1997) Let's ask the users. IEEE Software, 110-111. [PDF*]
  • Punter, T., Ciolkowski, M., Freimut, B. & John, I. (2003) Conducting on-line surveys in software engineering. 2003 International Symposium on Empirical Software Engineering (ISESE'03). [PDF*]
  • Robson, C. (1994) Real world research: a resource for social scientists and practitioner-researchers. Blackwell publishers.
  • Teague, R., De Jesus, K. & Nunes-Ueno, M. (2001) Concurrent vs. post-task usability test ratings. CHI '01 extended abstracts on Human factors in computing systems, p. 289-290. ACM DOI
  • Vinkki: katso myös Usability Questionnaires
Fokusryhmät (Focus groups)
  • Bloor, M., Frankland, J., Thomas, M. & Robson, K. (2001) Focus Groups in Social Research. London: Sage.
  • Bruseberg, A. & McDonagh-Philp, D. (2001) New product development by eliciting user experience and aspirations. International Journal of Human-Computer Studies, 55 (4), October 2001, 435-452. DOI.
  • Bruseberg, A and McDonagh-Philp, D. (2002) Focus groups to support the industrial/product designer: a review based on current literature and designers' feedback. Applied Ergonomics: Human Factors in Technology and Society, 33(1), 27-38. [PDF * (draft version)]
  • Dawson, S. & Manderson, L. (1993) A Manual for the Use of Focus Groups. HTML
  • Mallon, B. & Webb, B. (2000) Structure, causality, visibility and interaction: propositions for evaluating engagement in narrative multimedia. International Journal of Human-Computer Studies 53 (2), August 2000, 269-287. DOI.
  • Nielsen, J. (1997) The use and misuse of focus groups. HTML.
  • Rosenbaum, S., Cockton, G., Coyne, K., Muller, M. & Rauch, T. (2002) Panel: Focus groups in HCI: Wealth of information or waste of resources? Proceedings of CHI 2002, 702-703. ACM DOI
  • Sato, S & Salvador, T. (1999) Playacting and Focus Troupes: Theater techniques for creating quick, intense, immersive, and engaging focus group sessions. Interactions 6(5), September-October 1999, 35-41. ACM DOI
  • What Are Focus Groups? (1997) Julkaisussa What is a Survey? ASA Series. Alexandria: American Statistical Association. PDF
Haastattelumenetelmät (Interviews)
  • Bainbridge, L. (1999) Verbal reports as evidence of the process operator's knowledge. International Journal of Human-Computer Studies 51 (2), August 1999, 213-238. DOI.
  • Hirsjärvi, S. & Hurme, H. (2001) Tutkimushaastattelu- Teemahaastattelun teoria ja käytäntö. Yliopistopaino, Helsinki. Granum
  • Kemp, E.A. & Ots, S. (1998) Evaluating user interfaces using techniques from qualitative data analysis. 1998 International Conference on Software Engineering: Education & Practice. [PDF*]
  • McGraw, K.L. (1994) Knowledge Acquisition and Interface Design. IEEE Software, 90-92. [PDF*]
  • Nielsen, J. (1997) Let's ask the users. IEEE Software, 110-111. [PDF*]
  • Nielsen, J., Mack, R.L., Bergendorff, K.H. & Grischkowsky N.L. (1986) Integrated software usage in the professional work environment: evidence from questionnaires and interviews. Proc. CHI'86, 162-167. ACM DOI
  • Åborg, C., Sandblad, B., Gulliksen, J. & Lif, M. (2003) Integrating work environment considerations into usability evaluation methods -- the ADA approach. Interacting with computers 15 (3), June 2003, 453-471. DOI. Saatavilla myös PDF.
  • (haastattelun peruskirjoja & oppaita)
Käyttäjän kontekstissa
Tilannetutkimus (Contextual Inquiry)
  • Contextual Inquiry ­ © 1995-2000 James Hom.
  • Dray, S., Siegel, D., Feldman, E. & Potenza, M. (2002) "Why do Version 1.0 and not release it?" Conducting Field Trials of the Tablet PC. interactions, 9(2), 11 - 16, March/April, 2002. PDF
  • Jääskö, V. & Mattelmäki, T. (2003) Observing and probing. Int Conf of Designing pleasurable products and interfaces DPPI ’03, 126-131. ACM DOI [pdf*]
  • Kantner, L., Sova, D.H. & Rosenbaum, S. (2003) Alternative Methods for Field Usability Research, SIGDOC 2003, 68-72. ACM DOI PDF
  • Raven, M.E. & Flanders, A. (1996) Using contextual inquiry to learn about your audiences. Asterisk Journal of Computer Documentation 20:1 (1996). ACM DOI
  • Wixon, D.R., Ramey, J., Holtzblatt, K., Beyer, H., Hackos, J., Rosenbaum, S., Page, C., Laakso, S.A. & Laakso, K-P. (2002) Usability in practice: field methods evolution and revolution. Proc. CHI'02 ACM DOI
Etnografia (Ethnography)
  • Gilmore, D. (2002) Understanding and overcoming resistance to ethnographic design research. interactions 9(3), May-June 2002, 29-35. ACM DOI
  • HCIRN Ethnography
  • Hemmings, T., Crabtree, A., Rodden, T., Clarke, K. & Rouncefield, M. (2002) Probing the probes. In the Seventh Biennial Participatory Design Conference, pages 42-50. Available as PDF
  • Hughes, J.A., Randall, D. & Shapiro, D. (1993) From ethnographic record to system design. Some experiences from the field. Computer Supported Cooperative Work 1:3, 123-141. (SO:lla)
  • Hughes, J., King, V., Rodden, T. & Andersen, H. (1994) Moving out from the control room: Ethnography in system design. Proceedings of CHI 1994. ACM DOI
  • Karasti, H. (1997) Bridging the analysis of work practice and system redesign in cooperative workshops. Proc. DIS'97. ACM [PDF*]
  • Millen, D. R. (2000). Rapid ethnography: Time deepening strategies for HCI field research. Proceedings of DIS 2000, 280-286. ACM DOI PDF
  • O'Brien, J., Rodden, T., Rouncefield, M. & Hughes, J. (1999) At home with the technology: An ethnographic study of a set-top-box trial. ACM Transactions on Computer-Human Interaction, 6 (3). 1999. 282-308. ACM DOI
  • Rose, A., Shneiderman, B. & Plaisant, C. (1995) An applied ethnographic method for redesigning user interfaces. Proceedings of DIS'95, 115-122. ACM DOI HTML
  • Sperschneider, W. & Bagger, K. (2003) Ethnographic fieldwork under industrial constraints: Toward Design-in-Context. International Journal of Human-Computer Interaction 15:1, 41-50. [pdf*]
  • Weinberg, J.B. & Stephen, M.L. (2002) Participatory design in a human-computer interaction course: Teaching ethnography methods to computer scientists. SIGCSE 2002, 237-241. ACM DOI PDF
Toiminnan teoria
(Activity theory)
  • Bødker, S. (1989) A human activity approach to user interfaces. Human-Computer Interaction 4(3), 171-195. [PDF*]
  • Decortis, F., Noirfalise, S. & Saudelli, B. (2000) Activity theory, cognitive ergonomics and distributed cognition: three views of a transport company. Int. Journal of Human-Computer Studies, 53(1), July 2000, 5-33. DOI [PDF*]
  • Kaptelinin, V., Nardi, B.A. & Macaulay, C. (1999) The activity checklist: a tool for representing the "space" of context. interactions 6:4 (July-August 1999), 27-39. ACM DOI.
  • Kuutti, K. (1995) Activity Theory as a potential framework for human-computer interaction research. In Nardi, B. (ed.) Context and Consciousness: Activity Theory and Human Computer Interaction. Cambridge: MIT Press, 1995, 17-44. [pdf*]
  • Petersen, M.G., Madsen, K.H. & Kjær, A. (2002) Usability of everyday technology: Emerging and fading opportunities. ACM Transactions on Computer-Human Interaction 9(2), June 2002, 74-105. ACM DOI.
Asiantuntijan tekemät käytettävyysarviot
Erilaisten säännöstöjen avulla tehdyt asiantuntija-arviot
(Heuristic evaluation, Expert evaluation, Standards evaluation)
  • Garzotto, F., Matera, M. & Paolini, P. (2000) Model-based heuristic evaluation of hypermedia usability. Proc. AVI'98, L'Aquila, Italy, June 1998, ACM Press, 135-145. ACM DOI. [PDF*]
  • Levi, M.D. & Conrad, F.G. (1996) A heuristic evaluation of a World Wide Web prototype. Interactions 3(4), 50-61. ACM DOI. See also: HTML.
  • Muller, M.J., McClard, A., Bell, B., Dooley, S., Meiskey, L., Meskill, J. A., Sparks, R. & Tellam, D. (1995) Validating an extension to participatory heuristic evaluation: Quality of work and quality of work life. CHI 95 Conference Companion, 115-116. HTML.
  • Nielsen, J. & Molich, R. (1990) Heuristic evaluation of user interfaces. Proceedings of CHI 1990, 249-256. ACM DOI
  • Nielsen, J. (1994) Enhancing the explanatory power of usability heuristics. Proceedings of CHI 1994, 152-158. ACM DOI
  • Oppermann, R. & Reiterer, H. (1997) Software evaluation using the 9241 evaluator. Behaviour & Information Technology 16 (4-5), July 1997, 232-245. PDF.
  • Sutcliffe, A. (2002) Assessing the Reliability of Heuristic Evaluation for Website Attractiveness and Usability. 35th Annual Hawaii International Conference on System Sciences (HICSS'02), Volume 5. [PDF*]
  • Zazelenchuk, T. (2003) In search of the Holy Grail: Alternatives to Nielsen's heuristics. HTML.
Ilman käyttäjää tehtävät läpikäynnit
(mm. kognitiivinen läpikäynti, Cognitive Walkthrough)
  • Blackmon, M.H., Polson, P.G., Kitajima, M., & Lewis, C. (2002) Cognitive walkthrough for the web. Proceedings of CHI 2002, 463-470. ACM DOI.
  • Lewis, C., Polson, P., Wharton, C. & Rieman, J. (1990) Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces. CHI 90, ACM, Seattle, 235-242. ACM DOI.
  • Polson, P.G., Lewis, C., Rieman, J. & Wharton, C. (1992) Cognitive walkthroughs: a method for theory-based evaluation of user interfaces, International Journal of Man-Machine Studies, 36, 741-773.
  • Rieman, J., Franzke, M. & Redmiles, D. (1995) Usability evaluation with the Cognitive Walkthrough. CHI'95 Tutorial. HTML
  • Rizzo, A., Marchigiani, E. & Andreadis, A. (1997) The AVANTI project: prototyping and evaluation with a cognitive walkthrough based on the Norman's model of action. In Designing Interactive Systems (DIS +97) Conference Proceedings, 305-309. ACM DOI
  • Rowley, D.E. & Rhoades, D.G. (1992) The cognitive jogthrough: a fast-paced user interface evaluation procedure. Proc. CHI 1992, 389-395. ACM DOI.
  • Sears, A. (1997) Heuristic walkthroughs: Finding the problems without the noise. Int. Journal of Human-Computer Interaction, 9(3), 213-234. [PDF*]
  • Spencer, R. (2000) The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company. Proc. CHI 2000, 353-359. ACM DOI.
  • Wharton, C., Rieman, J., Lewis, C., & Polson, P. (1994) The cognitive walkthrough method: A practitioner's guide. In Nielsen, J. & Mack, R.L. (Eds.), Usability inspection methods, 105-140. New York, NY: John Wiley & Sons.
Osallistavat ryhmäläpikäynnit
(Pluralistic walkthrough)
  • Bias, R.G. (1994) The pluralistic usability walkthrough: Coordinated empathies. In Nielsen, J. & Mack, R.L. (Eds.), Usability inspection methods, 63-76. New York, NY: John Wiley & Sons.
  • Bias, R. (1991) Walkthroughs: Efficient collaborative testing. IEEE Software, 8, 5, 94-95. [PDF*]
  • Madsen, K.H. & Aiken, P.H. (1993) Experiences using cooperative interactive storyboard prototyping. Communications of the ACM 36:6. ACM DOI
  • Muller, M.J., Matheson, L., Page, C. & Gallup, R. (1998) Participatory heuristic evaluation. interactions, 5, 5, 13-18. ACM DOI.
  • Paterno, F. and Santoro, C. (2002). Preventing user errors by systematic analysis of deviations from the system task model Int. J. Human-Computer Studies (2002) 56, 225-245. DOI
  • Riihiaho, S. (2002) The pluralistic usability walk-through method. Ergonomics in Design. Human Factors and Ergonomics Society. 10, 3, 23-27. PDF
  • Wharton, C., Bradford, J., Jeffries, R. & Franzke, M. (1992) Applying cognitive walkthroughs to more complex user interfaces: experiences, issues, and recommendations. Proc. CHI'1992, 381-388. ACM DOI
  • Zack, L.W. (1994) Using electronic group decision-making technology for usability walkthroughs: an initial qualitative cost/benefit analysis. CHI 1994 Conference Companion, 273-274. ACM DOI
Automaattiset menetelmät
(Automatic Usability Evaluation)
  • Andre, T.S. (2000) Determining the effectiveness of the Usability Problem Inspector: A theory-based model and tool for finding usability problems. Ph.D. Thesis, Virginia Polytechnic Institute and State University. PDF
  • Byrne, M.D., Wood, S.D., Foley, J.D., Kieras, D.E. & Sukaviriya, P.N. (1994) Automating interface evaluation. Proc. CHI 1994, 232-237. ACM DOI
  • Hilbert, D.M. & Redmiles, D.F. (2000) Extracting usability information from user interface events. ACM Computing Surveys 32:4, December 2000, 384-421. ACM DOI.
  • Ivory, M. & Hearst, M. (2001) The state of the art in automating usability evaluation of user interfaces. ACM Computing Surveys 33(4), December 2001, 470-516. ACM DOI [PDF*]
  • Laskowski, S.J., Landay, J.A. & Lister, M. (2002) Automatic capture, representation, and analysis of user behavior CHI '02 extended abstracts, 922-923. ACM DOI [PDF*]
  • Lecerof, A. & Paternò, F. (1998) Automatic support for usability evaluation. IEEE Transactions on Software Engineering 24:10, October 1998, 863-888. IEEE. [PDF*]
Testaus
Käytettävyystestaus ja sen eri muodot: paritestaus, etätestaus
  • Beck, E., Christiansen, M., Kjeldskov, J., Kolbe, N. & Stage, J. (2003) Experimental evaluation of techniques for usability testing of mobile systems in a laboratory setting. OzCHI 2003, Brisbane, Australia. PDF
  • Cordes, R.E. (2001) Task-selection bias: A case for user-defined tasks. International Journal of Human-Computer Interaction 13(4), 411-419. [pdf*]
  • Dicks, R.S. (2002) Mis-usability: on the uses and misuses of usability testing. Proc. 20th annual int. conf. on Computer Documentation, 26-30. ACM DOI.
  • Dolan, W.R. & Dumas, J.S. (1999) A flexible approach to third-party usability. Communications of the ACM 42:5, May 1999, 83-85. ACM DOI
  • Dray, S., & Siegel, D. (2004) Remote possibilities?: international usability testing at a distance. interactions, 11(2), 2004 (March-April), 10-17. ACM DOI
  • Ebling, M. & John, B.E. (2000) On the contributions of different empirical data in usability testing. Symposium on Designing Interactive Systems 2000, 289-296. ACM DOI
  • Hammontree, M., Weiler, P. & Nayak, N. (1994) Remote usability testing. Interactions 1:4 (July 1994), 21-25. ACM DOI.
  • Holleran, P.A. (1991) A methodological note on pitfalls in usability testing. Behaviour & Information Technology 10, 5, 345-357.
  • Kahler, H., Kensing, F. & Muller, M. (2000) Constructive interaction and collaborative work: Introducing a method for testing collaborative systems. interactions 7:3 (May 2000), 27-34. ACM DOI
  • Rowley, D.E. (1994), Usability testing in the field: bringing the laboratory to the user. CHI 1994, 252-257. ACM DOI
  • Tognazzini, B. (2001) How to Deliver a Report Without Getting Lynched. HTML.
  • Virzi, R.A., Sokolov, J. & Karis, D. (1996) Usability problem identification using both low- and high-fidelity prototypes. Proc. CHI'96, 236-243. ACM DOI.
  • Waterson, S., Matthews, T. & Landay, J.A. (2002) In the lab and out in the wild: Remote web usability testing for mobile devices. In Proceedings of ACM CHI 2002 Conference on Human Factors in Computing Systems. Minneapolis, MN, April 2002. ACM DOI
Ääneenajattelu (Think Aloud Method)
  • Boren, M. & Ramey, J. (2000), Thinking aloud: Reconciling theory and practice. IEEE Transactions on Professional Communication, 43(3): 261-278. PDF
  • Ericsson, K.A. & Simon, H.A. (1993) Protocol analysis: Verbal reports as data. Revised edition. Cambridge, MA: The MIT Press. (laitoskirjasto).
  • Nielsen, J., Clemmensen, T. & Yssing, C. (2002) Getting access to what goes on in people's heads?: reflections on the think-aloud technique. In Nordichi 2002 - Tradition and Transcendence, Aarhus, Denmark, October 19-23, 2002. ACM DOI
  • van Someren, M., Barnard, Y., & Sandberg, J. (1994). The think aloud method. A practical guide to modelling cognitive processes. London: Academic Press. PDF
  • Van Waes, L. (2000) Thinking aloud as a method for testing the usability of websites: The influence of task variation on the evaluation of hypertext. IEEE Transaction on Professional Communication, 43, 4, 279-291. (pyydetty tekijältä paperikopio)
  • Van Waes, L. (2001) Testing usability tests for web sites: an evaluation of task variation in think aloud sessions. In Human-computer interaction INTERACT '01. Amsterdam, IOS Press, 2001, p. 771-772. (laitoskirjasto)
Katseenseuranta (Eye Tracking)
  • Benel, D.C.R., Ottens, D. & Horst, R. (1991). Use of an eye tracking system in the usability laboratory. Proceedings of the Human Factors Society 35th Annual Meeting. 461-465. Santa Monica, Human Factors and Ergonomics Society. (pyydä paperikopsu Päiviltä)
  • Card, S.K., Pirolli, P., Van Der Wege, M.M., Morrison, J.B., Reeder, R.W., Schraedley, P.K., & Boshart. J. (2001) Information scent as a driver of Web behavior graphs: results of a protocol analysis method for Web usability. CHI 2001, 498-505. ACM DOI (See also WebEyeMapper and WebLogger)
  • Cowen, L., Ball, L.J. & Delin, J. (2002), An Eye Movement Analysis of Web Page Usability, in Proceedings of HCI 2002, People and Computers XVI - Memorable Yet Invisible, 317-335. PDF (See also Cowen's thesis: PDF)
  • Ellis, S.H., Candrea, R., Misner, J., Craig, C.S., Lankford, C. & Hutchinson, T. E. (1998). Windows to the soul? What eye movements tell us about software usability. Proceedings of the 7th Annual Conference of the Usability Professionals Association, Washington, D.C., June 1998. [DOC * draft]
  • Goldberg, J.H., Stimson, M.J., Lewenstein, M., Scott, N. & Wichansky A.M. (2002) Eye tracking in Web search tasks: Design implications. Proceedings of the symposium on ETRA 2002: eye tracking research & application symposium, pp. 51-58. ACM DOI
  • Goldberg, J.H. & Wichansky, A.M. (2003) Eye tracking in usability evaluation: A practitioner's guide. In Hyönä, J., Radach, R. & Deubel, H. (Eds.), The mind's eye: Cognitive and applied aspects of eye movement research. Amsterdam, The Netherlands: North-Holland. (pyydä paperikopsu Päiviltä)
  • Jacob, R.J.K & Karn, K.S. (2003) Eye tracking in human-computer interaction and usability research: Ready to deliver the promises (Section commentary) In Hyona, J., Radach, R. & Deubel, H. (eds.) The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research, 573-605. Amsterdam, Elsevier Science, 2003. PDF
  • Nielsen, J. (2002) Eyetracking study of Web readers. Jakob Nielsen's Alertbox, May 14, 2000. HTML
  • Redline, C. & Lankford, C. (2001) Eye-movement analysis: A new tool for evaluating the design of visually administered instruments (paper and Web). American Association for Public Opinion Research Annual Meeting, Montreal, CA, May 2001. PDF
  • Schiessl, M., Duda, S., Thölke, A. & Fischer, R. (2003) Eye tracking and its application in usability and media research. In Journal "Sonderheft: Blickbewegung" in MMI-interaktiv Journal - Online Zeitschrift zu Fragen der Mensch-Maschine-Interaktion. ISSN 1439-7854. Gastherausgeber: Katharina Seifert & Matthias Rötting. 12.03.03, Ausgabe Nr. 6. PDF
Menetelmien vertailu ja muut teemat
Arvioijan vaikutus tuloksiin (Evaluator effect)
  • Andre, T.S., Hartson, H.R., Belz, S.M., & McCreary, F.A. (2001) The user action framework: A reliable foundation for usability engineering support tools. International Journal of Human-Computer Studies, 54, 107-136. PDF
  • Cockton, G. & Woolrych, A. (2002) Sale must end: Should discount methods be cleared off HCI's shelves? Interactions 9:5 September & October 2002, 13-18. ACM DOI (ks. myös siihen liittyvä kommentti Manning, H. (2002) Must the sale end? Interactions 9:6 November-December 2002, 56-ff. ACM HTML.)
  • Hartson, H.R., Andre, T.S., Williges, R.C. & van Rens, L. (1999). The user action framework: A theory-based foundation for inspection and classification of usability problems. In Bullinger, H.-J. & Ziegler, J. (Eds.) Human-computer interaction: Ergonomics and user interfaces. Proceedings of HCI International 99, Volume 1, 1058-1062. Mahwah, NJ: Lawrence Erlbaum Associates. PDF.
  • Hassenzahl, M. (2000) Prioritizing usability problems: data-driven and judgement-driven severity estimates. Behaviour & Information Technology 19(1) January-February 2000, 29-42.
  • Hertzum, M. & Jacobsen, N.E. (2001) The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 13(4), 421-443. PDF
  • Hertzum, M., Jacobsen, N.E. & Molich, R. (2002) Usability inspections by groups of specialists: perceived agreement in spite of disparate observations. Proc. CHI '02 extended abstracts on Human factors in computing systems ACM DOI
  • Jacobsen, N.E., Hertzum, M. & John, B.E. (1998) The evaluator effect in usability tests. Proceedings of CHI98, 255-256. ACM DOI
Osallistujien valinnan ja määrän vaikutus tuloksiin
  • Dray, S. & Siegel, D. (1999) Penny-wise, pound-wise: making intelligent trade-offs in planning usability studies. interactions 6:3, 25-30, May/June 1999. PDF
  • Dumas, J.S. & Redish, J.C. (1993) A Practical Guide to Usability Testing, Norwood N.J., Ablex. (luku 9: Deciding who should be participants, s.119-133)
  • Holleran, P. (1991) A methodological note on pitfalls in usability testing. Behaviour & Information Technology, 10, 5, 1991, 345-357.
  • Thomas, J.C. & Kellogg, W.A. (1989) Minimizing ecological gaps in interface design. IEEE Software 6(1), 78-86. [pdf*
  • Lewis, J.R. (1994) Sample sizes for usability studies: Additional considerations. Human Factors, 36, 2, 1994, 368-378.
  • Nielsen, J. (1994) Estimating the number of subjects needed for a thinking aloud test. Int. J. Human-Computer Studies 41, 385-397.
  • Nielsen, J. (2000) Why you only need to test with five users. Jakob Nielsen's Alertbox, March 19, 2000, HTML
  • Rosenbaum, S. (1987) Selecting the appropriate subjects: subject selection for documentation usability tasting. Proceedings of the 1987 IEEE International Professional Communication Conference.
  • Rubin, J. (1994) Handbook of usability testing: how to plan, design, and conduct effective tests. New York, Wiley. (luku 6: Selecting and acquiring participants, s.119-139)
  • Spool, J. & Schroeder, W. (2001) Testing web sites: Five users is nowhere near enough. Extended abstracts of CHI 2001, 285-286. ACM DOI. Also (PDF)
  • Virzi R.A. (1992) Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34, 4, 1992, 457-468.
  • Woolrych, A. & Cockton, G. (2001) Why and when five test users aren't enough. In Proceedings of IHM-HCI 2001 Conference Volume 2, 105-108. doc.
Menetelmien vertailu
  • Gray, W.D. & Salzman, M.C. (1998) Damaged merchandise? A review of experiments that compare usability evaluation methods. Human-Computer Interaction 13(3), 203-261. [pdf*]
  • Gray, W.D. & Salzman, M.C. (1998). Repairing Damaged Merchandise: A Rejoinder. Human-Computer Interaction 13(3), 325-335. [pdf*]
  • Hartson, H.R., Andre, T.S. & Williges, R.C. (2003) Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 15:1, 145-181. [pdf*]
  • Hughes, M. (1999) Rigor in usability testing. Technical Communication, 46, 4 (November 1999), 488-494. Saatavilla PDF.
  • John, B.E. & Marks, S.J. (1997) Tracking the effectiveness of usability evaluation methods. Behaviour & Information Technology 16:4/5, 188-202.
  • Lavery, D., Cockton, G. & Atkinson, M.P. (1997) Comparison of evaluation methods using structured usability problem reports. Behaviour & Information Technology, 16 (4/5), 246-266.
  • Redish, J., Molich, R., Bias, R.G., Dumas, J., Bailey, R. & Spool, J.M. (2002) Usability in practice: Formative usability evaluations - evolution and revolution. Proceedings of CHI 2002, 885-890.
  • Vredenburg, K., Mao, J.-Y., Smith, P.W. & Carey, T. (2002) A survey of user-centered design practice. Proc. CHI'02, 471 - 478. ACM DOI
  • Wixon, D. (2003) Evaluating usability methods: why the current literature fails the practitioner. interactions 10:4 (July 2003). ACM DOI
Erikoisryhmien käytettävyys; saavutettavuus (accessibility) ja sen arviointimenetelmät
  • Hawthorn, D. (2002) How universal is good design for older users? In ACM SIGCAPH Computers and the Physically Handicapped Issue 73-74 June-September 2002 (also in Proceedings of the 2003 conference on Universal usability, p. 38-45) ACM DOI
  • Hull, L. (2004) Accessibility: it's not just for disabilities any more. interactions, 11(2), 2004 (March-April), 36-41. ACM DOI
  • Korpela, J. Esteettömyysopas. html.
  • Paddison, C. & Englefield, P. (2003) Applying heuristics to perform a rigorous accessibility inspection in a commercial context, Proceedings of the 2003 conference on Universal usability, 126-133. ACM DOI.
  • Sloan, D., Gregor, P., Rowan, R. & Booth, P. (2000) Accessible accessibility. Proc. CUU 2000 First ACM Conference on Universal Usability, 96-101. ACM DOI
  • Sutcliffe, A., Fickas, S., Sohlberg, M.M. & Ehlhardt, L.A. (2003) Investigating the usability of assistive user interfaces. Interacting with Computers 15(4), August 2003, 577-602. DOI.
  • Theofanos, M.F. & Redish, J. (2003) Bridging the gap: between accessibility and usability. interactions 10:6, November+December 2003, 36-51. ACM DOI.
Erikoisryhmien kanssa tehtävä käytettävyysarviointi: case lapset
  • Druin, A. (1999) Cooperative Inquiry: Developing new technologies for children with children. Proceedings of CHi'99, 592-599. [pdf*]
  • Hanna, L., Risden, K. & Alexander, K. (1997) Guidelines for usability testing with children. interactions 1997, 4 (5), 9-14. [pdf*]
  • Kersten-Tsikalkina, M. & M. Bekker (2001) Evaluating usability and pleasure of children's products. In Proc. of the Int. Conf. on Affective Human Factors Design, Singapore, June 27 -29. [pdf*]
  • Markopoulos, P. & Bekker, M. (2003) On the assessment of usability testing methods for children. Interacting with Computers, 15:2, 227-243. DOI [pdf*]
  • Read, J.C., MacFarlane, S.J. & Casey, C. (2002) Endurability, engagement and expectations: Measuring children's fun. Proceedings of Interaction Design and Children, International Workshop, August 28-29 2002 Eindhoven, The Netherlands. [pdf*]
  • Van Kesteren, I., Bekker, M.M., Vermeeren, A.P.O.S. & Lloyd, P. (2003) Assessing usability evaluation methods on their effectiveness to elicit verbal information from children subjects. Conference on Interaction Design and Children, Preston, 1-3 July 2003, 41-49. ACM DOI [pdf*]
Ryhmäohjelmien käytettävyysarviointi
  • Baker, K., Greenberg, S. & Gutwin, C. (2002) Empirical Development of a Heuristic Evaluation Methodology for Shared Workspace Groupware. Proc. CSCW'02, 96-105. ACM DOI
  • Pinelle, D. & Gutwin, C. (2002) Groupware walkthrough: Adding context to groupware usability evaluation. Proceedings of CHI 2002, 455-462. New York, NY: ACM. ACM DOI.
  • Pinelle, D., Gutwin C. & Greenberg, S. (2003) Task analysis for groupware usability evaluation: Modeling shared-workspace tasks with the mechanics of collaboration. ACM Transactions on Human Computer Interaction 10:4, December 2003, 281-311. ACM DOI. PDF.
  • Pinelle, D. & Gutwin, C. (2000) A review of groupware evaluations. IEEE WETICE 2000. PDF. (Katso myös Univ. of Saskatchewan muita julkaisuja.)
  • Steves, M.P., Morse, E., Gutwin, C. & Greenberg, S. (2001) A comparison of usage evaluation and inspection methods for assessing groupware usability. Proc. SIGGROUP 2001, 125-134. ACM DOI [PDF*]
Käytettävyystutkimuksen etiikka
  • Burmeister, O.K. (2001) HCI Professionalism: Ethical concerns in usability engineering. In Proc. 2nd Australian Institute of Computer Ethics Conference, Canberra, Australia, December 2000. work. ACM DOI [PDF*]
  • Ethics in HCI and Usability. http://www.stcsig.org/usability/topics/ethics.html
  • Friedman, B. & Nissenbaum, H. (1996) Bias in computer systems. ACM Transactions on Information Systems 14:3, 330-347. ACM DOI [PDF*]
  • Friedman, B. & Kahn, P.H., Jr. (2003) Human values, ethics, and design. In Jacko, J.A. & Sears, A. (Eds.), The Human-Computer Interaction Handbook, 1177-1201. Mahwah, NJ: Lawrence Erlbaum Associates. [PDF] [PDF*]
  • Mackay, W. (1995) Ethics, Lies and Videotape. Proc. CHI'95. Available as PDF
  • See also Ethics in HCI
  • Selected papers from the second Australian Institute conference on Computer ethics - Volume 1 (2000). Saatavilla sivulta HTML.
  • ACM Code of Ethics, 1992.
  • American Psychological Association's Ethical Principles of Psychologists and Code of Conduct (2002). See HTML. Linkki tarkistettu 14.1.2004.
  • CPA Ethical Standards related to the research participants Bill of Rights (2003) HTML

Seminaarin kotisivu Käytettävyystutkimuksen menetelmät -seminaari
Saila Ovaska (ov@cs.uta.fi)
Sivua viimeksi päivitetty 27. toukokuuta 2004.
Tietojenkäsittelytieteiden laitos
Tampereen yliopisto