Jeremy Smith

Friday, September 14, 2007

This blog has defiantly taken a back seat while I complete my thesis. There are some big things planned for it, so be ready when the time comes! In the mean time check out my mobile-blog at jeremysmith.tumblr.com . It contains photos uploaded live from my mobile when I’m out and about, and also some funny snippets from the web. It’s updated regularly (unlike this blog).

Cheers!

Jeremy

Sunday, June 10, 2007

The History of Human Computer Interaction

Computer scientists such as Douglas Engelbart, Ivan Sutherland and Vannevar Bush developed the first concepts that have underpinned and permanently shaped human-computer interaction and computing in general. These ubiquitous inventions such as the mouse, graphical user interfaces, icons, windows, graphical manipulation, hypertext and the internet, were all developed by individuals like these at the forefront of technology for their time and clearly had a phenomenal, albeit unintentional, influence on the future field of HCI.

More recently, and at the more theoretical end of the HCI spectrum are researchers such as Donald Norman, Jakob Neilson and Cathleen Wharton. These individuals have stood upon the shoulders of these aforementioned forefathers and constructed heuristics, methodologies and procedures that guide the design and assessment of systems to help transform the field of HCI it into a modern, commercially viable mature discipline that exists today. However, without the prior work of the earlier pioneers, tools such as cognitive walkthroughs, usability heuristics and web design would not be necessary, let alone exist.

Choosing individuals or groups that have had the most profound influence on HCI depends upon the angle of examination and the importance one places upon a pioneering or academic view. If one is considering the most influential HCI theory, then perhaps the more theoretical Norman’s and Nielson’s later in the century would more appropriate, however as their work is reliant upon the inventions and developments of earlier individuals it is more correct to examine the pioneering individuals for their profound influence on the field of HCI.

Listed below in order of influence are two individuals and one research team that I believe have influenced the field of HCI most profoundly; Douglas Engelbart, Ivan Sutherland and the team behind the Xerox Star.

Douglas Engelbart

"I don't know what Silicon Valley will do when it runs out of Doug's [Engelbart’s] ideas."

- Alan Kay

Douglas Engelbart is, in the author’s opinion, the single individual who has most profoundly influenced the field of Human Computer Interaction. He is credited with the invention of the computer mouse (which on its own is influential enough) (Engelbart, 1970), the On-Line System (NLS), the ARPANET (the precursor to the internet), several graphical user interface components and the first practical use of hypertext. Furthermore, according to Alan Key (1987) Engelbart was one of the first software engineers to actually study his users. Although a widely accepted as a critical HCI practice nowadays, this was not a common method at all in the 1960’s. While the end result of his user studies was the development of an ergonomic lap-board, consisting of a keyboard, mouse and chord keyboard, perhaps the most important outcome was the conception of a rudimentary user centered design process that paved the way for user-centered designers such as Donald Norman and his book, The Psychology of Everyday Things (1986). Below are two of Engelbart’s most influential inventions and the influence they have had upon the field of HCI.

The Mouse

Originally patented as an ‘X-Y Position Indicator for a Display’ (Engelbart, 1970) and later nicknamed the mouse, due to the rodent-like tail at the end (Voelcker & Perry, 1989), the computer mouse has become ubiquitous and inseparable to modern computer systems. After development of the mouse, several engineers on Engelbarts team left the group and began working at Xerox PARC, a private research facility in Palo Alto and took with them their ideas, which was later incorporated into the Xerox Star and consequently the Apple Lisa. Most major operating systems rely upon this input device to some extent and until recently has remained fairly similar to the original design. There is no way Engelbart could have realised the impact and ubiquity his invention of an ‘X-Y position indicator’ would have on the future of computing, or how it would shape all major graphical user interfaces such as Microsoft Windows, MacOS, KDE and Gnome Linux desktop environments, which all rely heavily upon the mouse.

On-Line System

Engelbart was the main driver behind the construction of the oN-Line System (NLS) at the Augmentation Research Center (ARC) at the Stanford Research Institute (SRI) during the 1960s which was presented to an audience in San Francisco on December 9, 1968 (Stanford University, 2007). The NLS is considered to be the first practical use of hyper-text, the mouse and screen windowing (Myers, 1998). This demonstration has been informally referred to as ‘the mother of all demonstrations’ (Wikipedia Collaborators, 2007) because of the fact that it demonstrated so many pioneering and influential developments, and is widely viewed and circulated today on the internet. The NLS was jointly funded by ARPA (Advanced Research Projects Agency), NASA (National Aeronautics and Space Administration) and the United States Air force. (Engelbart Video Demonstration, 1968)

Engelbart recognised that collaboration was a fundamental component of work, and incorporated certain remote collaboration tools into the prototype. Interestingly, in a recorded video lecture presented by Allan Kay in 1987 on the history of computing, Kay points out that that this idea of collaboration is still struggling to come to the vision that Engelbart demonstrated and envisioned in the 1960’s, Kay is obviously (and understandably) unaware of the looming Internet boom less than a decade away.

Although hypertext had already been developed in ARPANET project, the NLS was the first demonstration of the practical combination of hyper-text, mouse and screen, and is therefore considered the major precursor to the internet. The influence the Internet has had on the field of HCI is indisputable, and has formed a foundation of research for many HCI researchers such as Jakob Nielsen.

Criticisms of Engelbart

Clearly, Engelbart has played a profound role in influencing the field of HCI through the invention of the mouse, and components of the Internet. However, his career and research has not been without its setbacks and errors of judgments. Unfortunately and through no real fault of his own, the mouse was patented in 1970 (Engelbart, 1970) and expired before they became widely used on the Apple Lisa, denying any royalties he may have deserved.

Alan Kay, while having the upmost regard for Engelbarts work, points out a few flaws and inconsistencies within Engelbarts work in a recorded lecture presented in 1987. Most of the criticisms orientate around the fact that, while Englebart placed a large emphasis on minimizing the movement between the mouse and keyboard, certain aspects of the system required extended swapping between the two. For example, to move a word in Engelbarts text editor, one had to first select the action to would perform from the keyboard. They system would then wait for the mouse input to select the word then use the keyboard to press accept, then complete the actual movement. This four step process was quite inefficient, especially when Engelbart had put so much effort into minimizing keyboard/mouse inter-movement (Kay, 1987). However this is not a big criticism, and may have been a response to the inaccurate analogue mouse (Kay, 1987).

Some members of Engelbarts team became alienated and moved to Xerox PARC around 1976 (Voelcker & Perry, 1989), possibly due to disagreements about the future of computing. While Engelbart incorrectly believed the future of computing would be orientated around a two-tier client-server architecture, while the younger programmers believed in the future of the personal computer. It was at Xerox PARC, where the Xerox Star was developed (Miller & Johnson, 1995), which used many of Engelbarts ideas and was a revolution in HCI. This split potentially could have lessened the influence on HCI of Engelbart and his team, who may have instead continued on their pioneering pathway, developing other HCI components. Instead, over time Engelbart became relatively obscure while other individuals and teams, such as Xerox PARC became more influential.

Ivan Sutherland

As part of a PHD thesis, Ivan Sutherland created the first graphical image manipulation program, titled Sketchpad (Sutherland, 1963) which was operated through a lightpen[1]. Although there was little immediate fallout from the thesis, Sketchpad is now widely understood to be as the grandfather of CAD and the graphical user interface (Bissell, 1990). Sketchpad’s design has been a primary influence on an entire generation of research into user interfaces (Perry & Voelcker, 1989) and many components of the program, such as tiled windows, are still used in graphical user interfaces today.

Sketchpad pioneered the creation and manipulation of graphic images directly on a CRT, rather than through the input of codes and algorithms, as previously required with existing software (Kay, 1987). The program enabled the user to draw lines, points and circular arcs, input constraints, zoom, and store primitive shapes. The program also pioneered the use of ‘objects’ and ‘instances’. An instance was a copy of the master object, which if updated would be reflected upon all instances (Miller & Johnson, 1996). This idea is widely used today in mainstream programs such as Adobe Illustrator and Adobe Flash (Moock, 2003). Furthermore, Sketchpad was the first program to use an object oriented software system and non procedural programming language (Kay, 1987), in itself an important development.

The Xerox P.A.R.C team behind the Xerox Star

Built with the ideology that users are interested in getting their work done and not about how the computer works, the Xerox Star was unfortunately not as successful as it was influential (Johnson Roberts Verplank Smith Irby Beard & Mackey, 1989). And although it has become relatively obscure over the years, concepts such as the desktop metaphor and icons have transcended into HCI and computers today, originating from when Apple borrowed many ideas from The Star to produce their Lisa model (Johnson, Roberts Verplank, Smith, Irby, Beard & Mackey, 1989)

One of the reasons the Xerox Star was so profoundly influential within the discipline of HCI was the fact that it was the first computer constructed based upon close examination the user interface, where paramount importance was placed upon the users conceptual model of the system, designing the interface even before the functionality of the system was fully decided (Smith, et. al., 1982). This was demonstrated in the fact that the team dedicated about 30 work years to it over the Star’s development life cycle (Smith et. al., 1982).

The Xerox Star, which went on sale in 1981, consisted of a bitmapped display, desktop metaphor, graphical controls, WYSIWYG[2] applications, multiple application windows and a mouse pointer (courtesy of Engelbart) (Miller & Johnson, 1996) that were all unique in the marketplace at the time (Johnston et al, 1989). Although The Star was not an overwhelming commercial success, the success of the graphical user interface components is reflected in the fact that they are now de facto standard in most graphical user interfaces today.

Both Engelbart and Sutherland’s work had a profound influence on the development of the Xerox Star in several ways. Firstly, Sutherland’s earlier Sketchpad system heavily influenced the Star’s user interface, such as through the use of tiled windows, as well as its graphics applications (Miller & Johnson, 1996), while the NLS, developed by Engelbart provided ideas such as interactivity, full-screen, CRT display, and the use of a mouse (Miller & Johnson, 1996).

It is a very hard task deciding upon an individual or team that has had the most profound influence on the field of HCI. Although there is no definitive answer, one can be certain that the early pioneers, such as Engelbart and Sutherland, paved the way for others later on in the century, and without them the field of HCI would be profoundly different.




References


Bissell, D., (1990) The father of computer graphics, Byte 1990

Who We Are How We Think. What we do, http://www.bootstrap.org, Accessed on 5 June 2007

Engelbart, D., (1988) Computer-supported cooperative work: a book of readings pp. 81 - 105

Engelbart D., (1962) Augmenting Human Intellect: A Conceptual Framework. Summary Report AFOSR-3223 under Contract AF 49(638)-1024, SRI Project 3578 for Air Force Office of Scientific Research, Stanford Research Institute, Menlo Park, Ca.,

Englebart, D. (1970). U.S. Patent No. 3.541.541. Palo Alto, California: U.S. Patent and Trademark Office.

Engelbart's Unfinished Revolution , A Symposium at Stanford University December 9, 1998 online at http://unrev.stanford.edu/introduction/introduction.html Accessed on 5 June, 2007

Engelbart, D (1968), ‘The Demo’ From Stanford University Video Archive, online at http://sloan.stanford.edu/mousesite/1968Demo.html accessed on 4 June 2007

Johnson, J., Roberts, T., Verplank, W., Smith, D., Irby, C., Beard, M., and Mackey K. (1989). The Xerox Star: A retrospective. IEEE Computer 22(9), 11-26

Kay, A (1987) Doing with Images Makes Symbols: Communicating With Computers Pt 1, University Video Communications.

MIT Inventor of the Week Series (2001) Online at http://web.mit.edu/invent/iow/sutherland.html Accessed on 4 June, 2007.

Myers, B (1998) A Brief History of Human-Computer Interaction Technology,

Miller, L., Johnson, J., (1996) The Xerox Star: An Influential User Interface Design, Human Computer Interaction Interface Design, pp 70-100 Interactions March-April

Moock, C., (2003) ActionScript for Flash MX: The Definitive Guide O’Reilly

Nielsen, J., (1995) Multimedia and Hypertext: the Internet and Beyond. Academic Press Professional, Boston.

Norman, D., (1988) The psychology of everyday things, Basic Book

Perry, T., Voelcker, J., (1989) “Of mice and menus: Designing the user-friendly interface” , from IEEE Spectrum 1989 A history of early GUIs, full of interesting tidbits

Perry, T., and Voelcker, J. (1989) Of Mice and menus: Designing the user-friendly interface. IEEE Spectrum.

Redant, J., (2001) HCI review of the Xerox Star Online at: http://xeroxstar.tripod.com/ Accessed on 4 June 2007

Sutherland, I., (1963) Sketchpad, a man-machine graphical communication system, PhD thesis, Massachusetts Institute of Technology

The Mother of All Demos. (2007, May 2). In Wikipedia, The Free Encyclopaedia. Retrieved 00:34, June 6, 2007, from http://en.wikipedia.org/w/index.php?title=The_Mother_of_All_Demos&oldid=12 7616555

Rudisill, M., Lewis, C., Polson, P., McKay, D., (1996) Human Computer Interaction Interface Design, Morgan Kaufmann Publishers Inc.

Smith, D., Irby, C., Kimball, R., & Harslem, E., (1982) The Star user interface: an overview proceedings of AFIPS 1982

Stanford University, The Mouse Site, Douglas Engelbart 1968 Demo Online at http://sloan.stanford.edu/mousesite/1968Demo.html, Accessed on 5 June, 2007


[1] Light pens are light sensitive pen like devices that are operated directly upon CRT screens.

[2] WYSIWYG is an acronym for What You See Is What You Get, and refers to the representation of text on-screen in a form corresponding to its printed form

Sunday, April 29, 2007

Mobile Devices to Aid the Elderly

As the population ages, especially in the western world, society is witnessing a rising proportion of elderly individuals (Jorge, 2001; Abascal & Civit, 2001). It has been estimated that by 2020, one in four Europeans will be over 60 years of age (Mikkonen, Vayrynen, Ikonen, & Heikkila, 2002) and the global rate of elderly individuals to be one billion in 2020 (Mikkonen et. al, 2002). These individuals are not, as often assumed, confined to their dwellings or nursing homes, but are in fact active social members of the community (Jorge, 2001), living healthy lives past the ages of 70 (Mikkonen et. al, 2002). Daily life however, is often a challenge for individuals with age induced problems such as motor, vision and auditory disabilities. Furthermore, as the proportion of elderly individuals increases, the rate of age induced cognitive disabilities such as Alzheimer's disease are also rising (Jorge, 2001). Like other elderly citizens, these individuals have friends, hobbies, family and friends, travel around their community and carry out daily activities such as cooking and taking medication (Jorge, 2001). Unlike the rest of the population however, these people often require the support of a care giver.

Technological development is already being shaped to accommodate elderly users’ special requirements, and many devices and services are on offer to help them cope with everyday activities (Mikkonen et. al, 2002) generally within the home. However, as technology becomes increasingly sophisticated, the idea of mobile services to support the elderly in a wide variety of situations is becoming more feasible and attractive as a tool to increase the autonomy of elderly people (Abascal & Civit, 2001)

Both mobile devices and elderly individuals pose very interesting and challenging interaction-design issues that need to be carefully understood for widespread adoption, acceptance and use. This literature review will examine three research papers in depth with the goal of developing a wider understanding of mobile devices, what they have to offer elderly people, and the associated challenges with this technology and this particular user demographic.

Three papers have been carefully selected, not only for their varying methodologies and angles, but for their similarities, which are manifested in a variety of different ways, some intentional and some not. Jorge (2001) focuses upon the issues and methodologies required in the development of situation aware mobile devices for elderly citizens. He argues that due to varying illnesses, disabilities and personality traits, it is almost impossible to design for each individuals unique needs. The solution, according to Jorge (2001), is the use of adaptive social assistants that change intelligently to the needs of elder users based upon information such as context and location. Abascal and Civit (2001) examine five self-described contexts that form the basis of a requirements analysis and subsequent discussion. Finally, Mikkonen, Väyrynen, Ikonen & Heikkilä (2002) conduct a participative two part study and determine four concepts that are analyzed in a participative manner by elderly citizens.

User centered design and participative design research methodologies were widely used and discussed in all three papers. It seems that Abascal & Civit's (2001) primary objective was almost to urge the “incorporation of older people’s needs to the design process”. They argue that the technology will only be successful if older people’s opinions are taken into consideration throughout the whole design process. They propose five situations in which to examine in order to identify elderly users requirements for mobile telecommunication. It is unclear as to how Abascal & Civit determine these five situations, and the article would be far more polished if the logic behind this was described. While in agreement with Abascal & Civit (2001), Mikkonen et. al. (2002) go one step further, by actually conducting an participatory ideation session with an elderly study group. The study consisted of five predefined phases based upon the OPERA framework: Own Suggestions, Pairs suggestions, Explanation, Ranking and Arranging. This resulted in the construction of four user produced technological concepts which were in turn evaluated on various levels by another elderly cohort. This was self titled a “User and Usages Study”. Although the Jorge (2001) article focused more closely upon the technical design aspects of mobile devices and elderly individuals, he too states that one needs to build on user centered design methodologies that combine task analysis with participatory design. He further states that careful studies of how the target population actually lives is critical in the success of this technology.

Surprisingly, as each paper was partly selected for its different angle, there were some very similar themes evident in the findings and discussions throughout all three papers. Of the four concepts produced and examined by Mikkonen et. al. (2002), a reminder device, that kept track of bills, medication and other information was the most technically feasible and received strong support from the elderly respondents. This is in line with Jorge (2001) who found that a key function a mobile assistant should perform would be to support daily routines that may suffer as a symptom of short term memory loss. Titled 'Memory Prosthesis', this hypothetical device would use location aware contextual information to provide unobtrusive reminders common to specific situations, reminiscent of the device elderly users outlined in the Mikkonen et. al. (2002) study.

The most popular, although currently unfeasible, device constructed by the participants in the Mikkonen et. al. (2002) study was a location aware Bluetooth-enabled security bracelet, titled “Mobile Wrist Care”. The device, already used within the home from a base station, would work through the mobile network and consequently outside the home. A panic button on the bracelet would connect through a Bluetooth connection to a larger device carried on the person or in the handbag. In the event of an emergency and the press of the panic button, coordinates of the individuals location, obtained from GPS positioning would be sent. Abascal & Civit (2001) discuss a very similar device with an inbuilt panic button, yet delve deeper and examine the situation in which the individual is too incapacitated to physically press the panic button. They outline a hypothetical device that could monitor vital health statistics such as heart rate, blood pressure and breathing rate, with the ability of notifying authorities if certain statistics reach a particular level.

Although phrased differently throughout each paper, way-finding was another finding that consistently echoed throughout the papers. Jorge (2001) outlined the case of how a blind individual could benefit from an adaptive tracking device. Although essentially based upon a standard GPS system, the adaptive social assistant would not only direct individuals, but it would use intelligent data to alert a caregiver if an individual strayed off a predefined route. The author also describes a particular case of boarding the correct bus and leaving it at the correct time. This directly aligns to the Mikkonen et. al. (2002) participative user study which results in the construction of a hypothetical service concept that enables elderly individuals to travel more independently on public transport. They describe a location aware portable terminal that informs the user when the correct bus arrives at the bus stop, even when the user cannot read the bus number. During the journey the mobile device informs the driver of the particular stop the passenger wishes to leave.

The prevalent theme throughout all the articles was one of aiding individual autonomy for as long as possible in an individual’s life. Each paper, with its different methods, goals and motives seemed to be touching upon this overall goal. While this is more obvious in some papers, such as the Abascal & Civit (2001) article titled “Universal Access to Mobile Telephone as a Way to Enhance the Autonomy of Elderly People” the other two papers suggested that having a society in which elderly individuals remain autonomous for as long as possible, it is not only better for the individual it is also better for the community.

Another theme of the papers was one of optimism of the future. Each paper was very optimistic of the role mobile technology will play in the future support of elderly individuals. The Mikkonen (2002) study, for example develops four different hypothetical technologies, of which three are currently technically infeasible. The Jorge (2001) paper revolves around the central idea that current static technologies are not suitable enough for the wide variety of elderly users and their particular uses. He proposes that “we should look to a not-so-distant future in which a variety of devices, such as wearable computers, intelligent environments and robots, monitor our medical status” and that an adaptive social assistant, currently not developed, could fill this user-service void. The Abascal & Civit (2001) paper also outlines several 'mixed technologies for future wireless telecommunications' such as a Mobile Text Telephony – a form of mobile Instant Message chat and Light Terminals – A mobile device capable of accessing the Internet. The article, currently 6 years old, is clearly showing its age with the current environment of relatively fast 3G mobile bandwidth. Nevertheless, the idea of widespread mobile Internet use by the elderly remains as futuristic as ever.

This brief literature synthesis, consisting of only three articles, just scrapes the surface as to the current state of research into mobile devices to aid the elderly, yet reflects the reoccurring themes that seem to resonate throughout most of the current research within this area. Some themes discussed and highlighted within this three paper literature review were:

  1. User centered design to understand the very complex user base.
  2. Aiding memory and supporting daily routines
  3. Contributing to personal security inside and outside the home
  4. Way-finding
  5. Autonomy for the elderly
  6. Looking towards the future.

Clearly, the potential benefits of mobile technology for the elderly are very high. However we are yet to see widespread adoption, use or even development of these aforementioned devices within the community. It is both heartening for the author to see the potential of future technology to help the elderly, yet somewhat disheartening to know that many of these individuals will not be around to take advantage of these devices in the future when they eventually become widely used.

References

Abascal, J., & Civit, A., 2001 Universal access to mobile telephony as a way to enhance the autonomy of elderly people, Proceedings of the 2001 EC/NSF workshop on Universal accessibility of ubiquitous computing: providing for the elderly pp 93-993

Jorge, J., 2001 Adaptive tools for the elderly: new devices to cope with age-induced cognitive disabilities Proceedings of the 2001 EC/NSF workshop on Universal accessibility of ubiquitous computing: providing for the elderly pp 66-70

Mikkonen, M., Vayrynen, S., Ikonen, V., and Heikkila, M. O., 2002 User and Concept Studies as Tools in Developing Mobile Communication Services for the Elderly Personal and Ubiquitous Computing 6:113-124