The History of Human Computer Interaction

Computer scientists such as Douglas Engelbart, Ivan Sutherland and Vannevar Bush developed the first concepts that have underpinned and permanently shaped human-computer interaction and computing in general. These ubiquitous inventions such as the mouse, graphical user interfaces, icons, windows, graphical manipulation, hypertext and the internet, were all developed by individuals like these at the forefront of technology for their time and clearly had a phenomenal, albeit unintentional, influence on the future field of HCI.

More recently, and at the more theoretical end of the HCI spectrum are researchers such as Donald Norman, Jakob Neilson and Cathleen Wharton. These individuals have stood upon the shoulders of these aforementioned forefathers and constructed heuristics, methodologies and procedures that guide the design and assessment of systems to help transform the field of HCI it into a modern, commercially viable mature discipline that exists today. However, without the prior work of the earlier pioneers, tools such as cognitive walkthroughs, usability heuristics and web design would not be necessary, let alone exist.

Choosing individuals or groups that have had the most profound influence on HCI depends upon the angle of examination and the importance one places upon a pioneering or academic view. If one is considering the most influential HCI theory, then perhaps the more theoretical Norman’s and Nielson’s later in the century would more appropriate, however as their work is reliant upon the inventions and developments of earlier individuals it is more correct to examine the pioneering individuals for their profound influence on the field of HCI.

Listed below in order of influence are two individuals and one research team that I believe have influenced the field of HCI most profoundly; Douglas Engelbart, Ivan Sutherland and the team behind the Xerox Star.

Douglas Engelbart

"I don't know what Silicon Valley will do when it runs out of Doug's [Engelbart’s] ideas."

- Alan Kay

Douglas Engelbart is, in the author’s opinion, the single individual who has most profoundly influenced the field of Human Computer Interaction. He is credited with the invention of the computer mouse (which on its own is influential enough) (Engelbart, 1970), the On-Line System (NLS), the ARPANET (the precursor to the internet), several graphical user interface components and the first practical use of hypertext. Furthermore, according to Alan Key (1987) Engelbart was one of the first software engineers to actually study his users. Although a widely accepted as a critical HCI practice nowadays, this was not a common method at all in the 1960’s. While the end result of his user studies was the development of an ergonomic lap-board, consisting of a keyboard, mouse and chord keyboard, perhaps the most important outcome was the conception of a rudimentary user centered design process that paved the way for user-centered designers such as Donald Norman and his book, The Psychology of Everyday Things (1986). Below are two of Engelbart’s most influential inventions and the influence they have had upon the field of HCI.

The Mouse

Originally patented as an ‘X-Y Position Indicator for a Display’ (Engelbart, 1970) and later nicknamed the mouse, due to the rodent-like tail at the end (Voelcker & Perry, 1989), the computer mouse has become ubiquitous and inseparable to modern computer systems. After development of the mouse, several engineers on Engelbarts team left the group and began working at Xerox PARC, a private research facility in Palo Alto and took with them their ideas, which was later incorporated into the Xerox Star and consequently the Apple Lisa. Most major operating systems rely upon this input device to some extent and until recently has remained fairly similar to the original design. There is no way Engelbart could have realised the impact and ubiquity his invention of an ‘X-Y position indicator’ would have on the future of computing, or how it would shape all major graphical user interfaces such as Microsoft Windows, MacOS, KDE and Gnome Linux desktop environments, which all rely heavily upon the mouse.

On-Line System

Engelbart was the main driver behind the construction of the oN-Line System (NLS) at the Augmentation Research Center (ARC) at the Stanford Research Institute (SRI) during the 1960s which was presented to an audience in San Francisco on December 9, 1968 (Stanford University, 2007). The NLS is considered to be the first practical use of hyper-text, the mouse and screen windowing (Myers, 1998). This demonstration has been informally referred to as ‘the mother of all demonstrations’ (Wikipedia Collaborators, 2007) because of the fact that it demonstrated so many pioneering and influential developments, and is widely viewed and circulated today on the internet. The NLS was jointly funded by ARPA (Advanced Research Projects Agency), NASA (National Aeronautics and Space Administration) and the United States Air force. (Engelbart Video Demonstration, 1968)

Engelbart recognised that collaboration was a fundamental component of work, and incorporated certain remote collaboration tools into the prototype. Interestingly, in a recorded video lecture presented by Allan Kay in 1987 on the history of computing, Kay points out that that this idea of collaboration is still struggling to come to the vision that Engelbart demonstrated and envisioned in the 1960’s, Kay is obviously (and understandably) unaware of the looming Internet boom less than a decade away.

Although hypertext had already been developed in ARPANET project, the NLS was the first demonstration of the practical combination of hyper-text, mouse and screen, and is therefore considered the major precursor to the internet. The influence the Internet has had on the field of HCI is indisputable, and has formed a foundation of research for many HCI researchers such as Jakob Nielsen.

Criticisms of Engelbart

Clearly, Engelbart has played a profound role in influencing the field of HCI through the invention of the mouse, and components of the Internet. However, his career and research has not been without its setbacks and errors of judgments. Unfortunately and through no real fault of his own, the mouse was patented in 1970 (Engelbart, 1970) and expired before they became widely used on the Apple Lisa, denying any royalties he may have deserved.

Alan Kay, while having the upmost regard for Engelbarts work, points out a few flaws and inconsistencies within Engelbarts work in a recorded lecture presented in 1987. Most of the criticisms orientate around the fact that, while Englebart placed a large emphasis on minimizing the movement between the mouse and keyboard, certain aspects of the system required extended swapping between the two. For example, to move a word in Engelbarts text editor, one had to first select the action to would perform from the keyboard. They system would then wait for the mouse input to select the word then use the keyboard to press accept, then complete the actual movement. This four step process was quite inefficient, especially when Engelbart had put so much effort into minimizing keyboard/mouse inter-movement (Kay, 1987). However this is not a big criticism, and may have been a response to the inaccurate analogue mouse (Kay, 1987).

Some members of Engelbarts team became alienated and moved to Xerox PARC around 1976 (Voelcker & Perry, 1989), possibly due to disagreements about the future of computing. While Engelbart incorrectly believed the future of computing would be orientated around a two-tier client-server architecture, while the younger programmers believed in the future of the personal computer. It was at Xerox PARC, where the Xerox Star was developed (Miller & Johnson, 1995), which used many of Engelbarts ideas and was a revolution in HCI. This split potentially could have lessened the influence on HCI of Engelbart and his team, who may have instead continued on their pioneering pathway, developing other HCI components. Instead, over time Engelbart became relatively obscure while other individuals and teams, such as Xerox PARC became more influential.

Ivan Sutherland

As part of a PHD thesis, Ivan Sutherland created the first graphical image manipulation program, titled Sketchpad (Sutherland, 1963) which was operated through a lightpen[1]. Although there was little immediate fallout from the thesis, Sketchpad is now widely understood to be as the grandfather of CAD and the graphical user interface (Bissell, 1990). Sketchpad’s design has been a primary influence on an entire generation of research into user interfaces (Perry & Voelcker, 1989) and many components of the program, such as tiled windows, are still used in graphical user interfaces today.

Sketchpad pioneered the creation and manipulation of graphic images directly on a CRT, rather than through the input of codes and algorithms, as previously required with existing software (Kay, 1987). The program enabled the user to draw lines, points and circular arcs, input constraints, zoom, and store primitive shapes. The program also pioneered the use of ‘objects’ and ‘instances’. An instance was a copy of the master object, which if updated would be reflected upon all instances (Miller & Johnson, 1996). This idea is widely used today in mainstream programs such as Adobe Illustrator and Adobe Flash (Moock, 2003). Furthermore, Sketchpad was the first program to use an object oriented software system and non procedural programming language (Kay, 1987), in itself an important development.

The Xerox P.A.R.C team behind the Xerox Star

Built with the ideology that users are interested in getting their work done and not about how the computer works, the Xerox Star was unfortunately not as successful as it was influential (Johnson Roberts Verplank Smith Irby Beard & Mackey, 1989). And although it has become relatively obscure over the years, concepts such as the desktop metaphor and icons have transcended into HCI and computers today, originating from when Apple borrowed many ideas from The Star to produce their Lisa model (Johnson, Roberts Verplank, Smith, Irby, Beard & Mackey, 1989)

One of the reasons the Xerox Star was so profoundly influential within the discipline of HCI was the fact that it was the first computer constructed based upon close examination the user interface, where paramount importance was placed upon the users conceptual model of the system, designing the interface even before the functionality of the system was fully decided (Smith, et. al., 1982). This was demonstrated in the fact that the team dedicated about 30 work years to it over the Star’s development life cycle (Smith et. al., 1982).

The Xerox Star, which went on sale in 1981, consisted of a bitmapped display, desktop metaphor, graphical controls, WYSIWYG[2] applications, multiple application windows and a mouse pointer (courtesy of Engelbart) (Miller & Johnson, 1996) that were all unique in the marketplace at the time (Johnston et al, 1989). Although The Star was not an overwhelming commercial success, the success of the graphical user interface components is reflected in the fact that they are now de facto standard in most graphical user interfaces today.

Both Engelbart and Sutherland’s work had a profound influence on the development of the Xerox Star in several ways. Firstly, Sutherland’s earlier Sketchpad system heavily influenced the Star’s user interface, such as through the use of tiled windows, as well as its graphics applications (Miller & Johnson, 1996), while the NLS, developed by Engelbart provided ideas such as interactivity, full-screen, CRT display, and the use of a mouse (Miller & Johnson, 1996).

It is a very hard task deciding upon an individual or team that has had the most profound influence on the field of HCI. Although there is no definitive answer, one can be certain that the early pioneers, such as Engelbart and Sutherland, paved the way for others later on in the century, and without them the field of HCI would be profoundly different.




References

Bissell, D., (1990) The father of computer graphics, Byte 1990

Bootstrap Institute, Who We Are How We Think. What we do, Online at http://www.bootstrap.org, Accessed on 5 June 2007

Engelbart, D., (1988) Computer-supported cooperative work: a book of readings pp. 81 - 105

Engelbart D., (1962) Augmenting Human Intellect: A Conceptual Framework. Summary Report AFOSR-3223 under Contract AF 49(638)-1024, SRI Project 3578 for Air Force Office of Scientific Research, Stanford Research Institute, Menlo Park, Ca.,

Englebart, D. (1970). U.S. Patent No. 3.541.541. Palo Alto, California: U.S. Patent and Trademark Office.

Engelbart's Unfinished Revolution , A Symposium at Stanford University December 9, 1998 online at http://unrev.stanford.edu/introduction/introduction.html Accessed on 5 June, 2007

Engelbart, D (1968), ‘The Demo’ From Stanford University Video Archive, online at http://sloan.stanford.edu/mousesite/1968Demo.html accessed on 4 June 2007

Johnson, J., Roberts, T., Verplank, W., Smith, D., Irby, C., Beard, M., and Mackey K. (1989). The Xerox Star: A retrospective. IEEE Computer 22(9), 11-26

Kay, A (1987) Doing with Images Makes Symbols: Communicating With Computers Pt 1, University Video Communications.

MIT Inventor of the Week Series (2001) Online at http://web.mit.edu/invent/iow/sutherland.html Accessed on 4 June, 2007.

Myers, B (1998) A Brief History of Human-Computer Interaction Technology,

Miller, L., Johnson, J., (1996) The Xerox Star: An Influential User Interface Design, Human Computer Interaction Interface Design, pp 70-100 Interactions March-April

Moock, C., (2003) ActionScript for Flash MX: The Definitive Guide O’Reilly

Nielsen, J., (1995) Multimedia and Hypertext: the Internet and Beyond. Academic Press Professional, Boston.

Norman, D., (1988) The psychology of everyday things, Basic Book

Perry, T., Voelcker, J., (1989) “Of mice and menus: Designing the user-friendly interface” , from IEEE Spectrum 1989 A history of early GUIs, full of interesting tidbits

Perry, T., and Voelcker, J. (1989) Of Mice and menus: Designing the user-friendly interface. IEEE Spectrum.

Redant, J., (2001) HCI review of the Xerox Star Online at: http://xeroxstar.tripod.com/ Accessed on 4 June 2007

Sutherland, I., (1963) Sketchpad, a man-machine graphical communication system, PhD thesis, Massachusetts Institute of Technology

The Mother of All Demos. (2007, May 2). In Wikipedia, The Free Encyclopaedia. Retrieved 00:34, June 6, 2007, from http://en.wikipedia.org/w/index.php?title=The_Mother_of_All_Demos&oldid=12 7616555

Rudisill, M., Lewis, C., Polson, P., McKay, D., (1996) Human Computer Interaction Interface Design, Morgan Kaufmann Publishers Inc.

Smith, D., Irby, C., Kimball, R., & Harslem, E., (1982) The Star user interface: an overview proceedings of AFIPS 1982

Stanford University, The Mouse Site, Douglas Engelbart 1968 Demo Online at http://sloan.stanford.edu/mousesite/1968Demo.html, Accessed on 5 June, 2007

---

[1] Light pens are light sensitive pen like devices that are operated directly upon CRT screens.

[2] WYSIWYG is an acronym for What You See Is What You Get, and refers to the representation of text on-screen in a form corresponding to its printed form