- Open Access
Turn off the graphics: designing non-visual interfaces for mobile phone games
Journal of the Brazilian Computer Society volume 15, pages 45–58 (2009)
Mobile phones are a widespread platform for ICT applications because they are highly pervasive in contemporary society. Hence, we can think of mobile gaming as a serious candidate to being a prominent form of entertainment in the near future. However, most games (for computers, console and mobile devices) make extensive use of the visual medium, which tends to exclude visually-impaired users from the play. While mobile gaming could potentially reach many visually-impaired users, who are very familiar with this technology, currently there seems to be only very few alternatives for this community. In an attempt to explore new interactive possibilities for such users, this work presents an initial study on non-visual interfaces for mobile phone games. It is based on Semiotic Engineering principles, emphasizing communication through aural, tactile and gestural signs, and deliberately excluding visual information. Results include a number of issues that can be incorporated to a wider research agenda on mobile gaming accessibility, both for the visually-impaired and sighted.
Souza CS.The semiotic engineering of human-computer interaction. Cambridge: The MIT Press; 2005.
Adams E, Rollings A.Fundamentals of game design. Upper Saddle River: Prentice Hall; 2006.
Baek J, Yun B. A sequence-action recognition applying state machine for user interface.IEEE Transactions on Consumer Electronics 2008; 54(2):719–726.
Barr P, Noble J, Biddle R. Video game values: human-computer interaction and games.Interacting with Computers 2007; 19(2):180–195.
Bystrom KE, Barfield W, Hendrix C. A conceptual model of sense of presence in virtual environments.Presence: Teleoperators and Virtual Environments 1999; 8(2):109–121.
Cai Y.The new frontier: portable and mobile gaming. Park & Associates Report. [on the internet]. Available from: 〈http://www.parksassociates.com/research/reports/tocs/2008/mobilegaming.htm〉. Access in: 16/08/2008.
Caillois R.Man, play and games. Champaign: University of Illinois Press; 2001.
Caldwell N. Theoretical frameworks for analyzing turn-based computer strategy games.Media International Australia Incorporating Culture and Policy 2004; (110):42–51.
Crawford C.The art of computer game design. [on the internet]. Washington: State University Vancouver; 1982. Available from: 〈http://www.vancouver.wsu.edu/fac/peabody/game-book/Coverpage.html〉. Access in: 16/08/2008.
Desurvire H, Caplan M, Toth JA. Using heuristics to evaluate the playability of games. In:Proceedings of the Conference on Human Factors in Computer Systems; 2004; Vienna. p. 1509–1512.
Drewes T, Mynatt E, Gandy M. Sleuth: an audio experience. In:Proceedings of The International Conference on Auditory Display; 2000; Atlanta. Atlanta: 2000.
Eco U.A theory of semiotics. Bloomington: Indiana University Press; 1976.
Ekman L, Ermi L, Lahti J, Nummela J, Lankoski P, Mäyrä F. Designing sound for a pervasive mobile game. In:Proceedings of the International Conference on Advances in Computer Entertainment Technology; 2005; Valencia. Valencia: ACM Press; 2005. p. 110–116.
Faust M, Yoo YH. Haptic feedback in pervasive games. In:Third International Workshop on Pervasive Gaming Applications; 2006; Dublin. Available from: 〈http://www.e56.de/download/HapticFeedbackInPervasiveGames.pdf〉. Access in: 11/08/2008
Friberg J, Gärdenfors D. Audio games: new perspectives on game audio. In:Proceedings of the International Conference on Advances in Computer Entertainment Technology; 2004; Singapore. Singapore: ACM Press; 2004. p. 148–154.
GAME Accessibility: gaming with a visual disability. [on the internet]. Available from: 〈http://www.game accessibility.com/index.php?pagefile=visual〉. Access in: 11/08/2008.
Gilbertson P, Coulton P, Chehimi F, Vajk T. Using “tilt” as an interface to control “no-button” 3-D mobile games.ACM Computers in Entertainment 2008; 6(3).
Glinert E, Wyse L. AudiOdyssey: an accessible video game for both sighted and non-sighted gamers. In:Proceedings of the 2007 conference on Future Play; 2007; Ontario. p. 251–252.
Huizinga J.Homo Ludens: a study of the play-element in culture. Boston: Beacon Press; 1966.
IGDA Game Accessbility SIG.Accessibility in games: motivations and approaches. [on the internet]. Available from: 〈http://www.igda.org/accessibility/IGDA_Accessibility_WhitePaper.pdf〉. Access in: 11/08/2008.
Iglesias R, Casado S, Gutierrez T, Barbero JI, Avizzano CA, Marcheschi S, Bergamasco M. Computer graphics access for blind people through a haptic and audio virtual environment. In:Proceedings of The 3rd IEEE International Workshop on Haptic, Audio, and Visual Environments and their Applications; 2004; Ottawa. p. 13–18.
Korhonen H, Koivisto EMI. Playability heuristics for mobile games. In:Proceedings of the 8th Conference on Human-computer interaction with mobile devices and services; 2006; Helsinki. p. 9–16.
Lakoff G, Johnson M.Metaphors we live by. Chicago; 1980.
Lévesque V.Blindness, technology and haptics. Technical Report TR-CIM-05.08. Montreal: Center for Intelligent Machines, Mc Gill University; 2005.
Liljedahl M, Papworth N, Lindberg S. Beowulf: an audio mostly game. In:Proceedings of the International Conference on Advances in Computer Entertainment Technology; 2007; Salzburg. p. 200–203.
Linjama J, Kaaresoja T. Novel, minimalist haptic gesture interaction for mobile devices. In:Proceedings of the NordiCHI 2004; 2004; Tampere. p. 457–458.
Lombard M, Ditton T. At the heart of it all: the concept of presence.Journal of Computer Mediated Communication 1997; 3(2). [on the internet]. Available from: 〈http://jcmc.indiana.edu/vol3/issue2/lombard.html〉. Access in: 11/08/2008.
Lumbreras M, Sánchez J. Interactive 3D sound hyperstories for blind children. In:Proceedings of the Conference on Human Factors in Computer Systems; 1999; Pittsburgh. Pittsburgh: ACM Press; 1999. p. 318–325.
Malone TW. Heuristics for designing enjoyable user interfaces: lessons from computer games. In:Proceedings of the Conference on Human Factors in Computer Systems; 1982; Gaithersburg. p. 63–68.
Murphy E, Kuber R, Strain P, McAllister G, Yu W. Developing sounds for a multimodal interface: conveying spatial information to visually impaired web users. In:Proceedings of The International Conference on Auditory Display; 2007; Montréal. p. 26-29.
Myers D.The nature of computer games: play as semiosis. New York: Peter Lang; 2003.
Ouhyoung M, Tsai WN, Tsai MC, Wu JR, Huang CH, Yang TJ. A low-cost force feedback joystick and its use in PC video games.IEEE Transactions on Consumer Electronics 1995; 41(3):787–794.
Nokia.Nokia N95 specification. [on the internet]. Available from: http://www.forum.nokia.com/devices/N95〉. Access in: 11/08/2008
Peirce CS.The Essential Peirce: selected philosophical writings. In: Houser N, Kloesel CJW. (Orgs.). Bloomington: Indiana University Press; 1998.
Pirhonen A, Murphy E, McAllister G, Yu W. Non-speech sounds as elements of a use scenario: a semiotic perspective. In:Proceedings of The International Conference on Auditory Display; 2007; London. p. 20–23.
Ravaja N, Saari T, Turpeinen M, Laarni J, Salminen M, Kivikangas M. Spatial presence and emotions during video game playing: does it matter with whom you play?Presence: Teleoperators and Virtual Environments 2006; 15(4): 381–392.
Salen K, Zimmermann E.Rules of play: game design fundamentals. Cambridge: The MIT Press; 2004.
Ur-Rehman S, Liu L, Li H. Vibration soccer: tactile rendering of football game on mobiles. In:Proceedings of the Next Generation Mobile Applications, Services and Technologies; 2007; Cardiff. Cardiff: IEEE Press; 2007. p. 9–13.
Vanderheiden G. Fundamental principles and priority setting for universal usability. In:Proceedings of the ACM Conference on Universal Usability; 2000; Arlington. Arlington: ACM Press; 2000. p. 32–37.
VIBETONZ system. [on the internet]. Available from: 〈http://www.vibetonz.com〉. Access in: 11/08/2008
Vorländer M.Auralization: fundamentals of acoustics, modelling, simulation, algorithms and acoustic virtual reality. Cambridge: Springer; 2007.
Witmer BG, Singer MJ. Measuring presence in virtual environments: a presence questionnaire.Presence: Teleoperators and Virtual Environments 1998; 7(3):225–240.
Zaphiris P, Ang CS. HCI issues in computer games.Interacting with Computers 2007; 19(2):135–139.
About this article
Cite this article
Valente, L., de Souza, C.S. & Feijó, B. Turn off the graphics: designing non-visual interfaces for mobile phone games. J Braz Comp Soc 15, 45–58 (2009). https://doi.org/10.1007/BF03192576
- Mobile non-visual games
- audio games
- haptics and gestures
- visually-impaired users
- semiotic engineering