I’ve been over at http://www.cogsci.ucsd.edu/~coulson/203/ reading some of the stuff Seana Coulson has been assigning to her Cog Sci grad students. This particular blog entry takes the discussion that Coulson, Grady and Oakley published in 1999 on metaphor theory and “blending” and uses it to think about the way we design user experiences.

The metaphors in our designs shape how users experience our UIs.

Few of our users are going to put up with an interface that “doesn’t make sense”—but if we’re giving them something new, how can they possibly understand how to use it? What convinces them that our UIs make sense? How could anything make sense if you’ve never seen it before and it’s unlike anything else?

Well, everything is like something else. Human beings are excellent at taking the alien and making it known. (Those things that are too uncomfortably foreign stay at arm’s length.)

Our job as designers is to take some domain a user understands and to get them to see that our interface is kind of like that. Our current metaphors for e-mail are “e-mail is like real mail”; “your e-mail service gives you a little office with an inbox and a trash can”. If we wanted to depart from current metaphors, we’re still going to have to make it like something else users understand. (Like a bulletin board outside your door? Like a soiree where friends drop by?)

We try to systematically project the language and imagery of one domain into another, hoping that the user will be able to understand that a lot inferences they take for granted in the source domain will also work in the target domain.

Another example you see on various web pages (note the rich metaphor of “the Web is like a book”; we often have more than one metaphor): “Enter this site”. “Enter” is a word used for physically crossing from one type of space into another (like a room). We’re saying here, “You are on the outside, you can come inside”, and more basically, “You move through the Web like you move through the real world. There are ‘sites’ (locations) and you can go into them and go out of them. You can explore and move around just as you do in the real world.”

How things go wrong

There are several possible breakdowns:

·         We can choose a “source domain” that users don’t actually know all that well. (Think of what would happen if users didn’t know about postal service and were trying to use our e-mail services.)

·         We can choose a source domain that doesn’t really map to the target domain. (But note that psycholinguistic results show that people are more likely to see a metaphor where there is greater semantic distance between the elements. That is, they are probably less likely to get confused if you’re saying “This website is like a garden” than if you’re saying “This website is like an Excel spreadsheet.”)

·         Users can overextend the metaphor and try to do things that do make sense in the source domain, but which don’t in the target domain. (“I better search for a stamp so I can send this e-mail.”)

More metaphor fundamentals

Design is never perfect. And part of what defines metaphors is that they involve a (temporary) suppression of important information about a particular domain. When we talk about the nation as a ship, we don’t actually mean that the nation has a mast and is sailing on the open seas.

When a metaphor works, a bunch of information is ignored—in the example of e-mail is like real mail, you ignore the fact that there aren’t stamps and that a man doesn’t actually come and pick up your e-mail to deliver it to a friend. But if we want to build in an idea of stamps or mailmen, those are right there and we can extend the metaphor.

That’s an important thing about metaphors: once we evoke a basic metaphor, we suddenly have primed all sorts of other ones. For example, there are all sorts of places you can enter, but a porn site might get you to think that “Enter the site” has a particularly transgressive boundary to cross. You could easily make the site familiar to physical environments that are like that: an adult bookstore at the edge of town, a sex labyrinth in Palm Springs.

(Note that these sorts of obvious designs come off as cheesy and amateurish. The real art to UI design is not being too heavy-handed. Early on practitioners realized that using a single metaphor was awfully clunky and limiting. Note also that I’ve never been to Palm Springs. But I’ve heard stories. Hedge mazes.)

Analyzing UIs

To analyze a UI, you may want to figure out what metaphors it’s conjuring up and which of them are the most basic. Most metaphors use similarity or analogy, but there are some primary metaphors that come out of actual physical experiences. For example, MORE IS UP comes from actually adding stuff to piles and seeing that there is more and that the pile grows. (See also Metaphors We Live By, Lakoff and Johnson 1980.)

Reference and summary

Grady, J., Oakley, T. & Coulson, S. (1999).  Conceptual Blending and Metaphor. In R. Gibbs (Ed.) Metaphor in Cognitive Linguistics.  Amsterdam & Philadelphia: John Benjamins.

Summary of the article itself: Many have thought of ‘conceptual metaphor theory’ and ‘blending theory’ as rivals. They are better seen as complements. CMT looks at generalizations across a broad range of metaphoric expressions; blending looks at particulars of individual examples. Thus CMT looks at long-term memory, while blending looks at on-line processes. “Consequently, metaphor theory will continue to address such questions as which concepts are conventionally associated with each other, how and why such conventional associations arise, and how cross-domain mappings are structured.” Blending may be able to explain how semantic properties of grammatical constructions combine with the lexical semantics of the words used in their instantiations.

Tyler’s “to read” list

On the role of distinct conceptual blends as experts and novices interact with Web browsers:

Maglio, P.P. & Matlock, T. 1998. "Metaphors we surf the web by". Workshop on Personalized and Social Information Space, Stockholm, Sweden.

One of our designers had some questions about whether young people are less distracted by IMs. Here is a reply from Microsoft Researcher Mary Czerwinski:

 

No, that’s not it.  Workers are definitely interrupted, but younger ones are more fluid in their ability to easily and effortlessly switch back and forth.  There is a cost, but they’ve learned to minimize it to a certain degree.

 

What I saw was that older users could focus on a single task much better and were loathe to switch until they were finished.

 

Does that make sense?

Sent: Monday, July 24, 2006 3:18 PM
To: Mary Czerwinski; Tyler Schnoebelen
Subject: RE: Some notes on interruption (for users, co-workers, etc)

 

Hi, Mary --

Thanks for the reply!

 

Would I be wrong in saying that the distinction you're making is between task interruption and task obstruction? 

 

--dae

From: Mary Czerwinski
Sent: Monday, July 24, 2006 3:13 PM
To: Tyler Schnoebelen
Subject: RE: Some notes on interruption (for users, co-workers, etc)

Hi folks,

 

Nice to meet you.  First, you should look at my real home page: http://research.microsoft.com/users/marycz as the other should be deleted, it’s old.

 

Yes, I have seen age effects in my research, but not in the direction you may be thinking!  Older folks are more likely to ignore IMs when they are focused on a task, and this is not at all true for younger folks.  I would sometimes have to interrupt an older user to check to see if they had any IMs, as they were so focused with their work they wouldn’t even see or hear them.  

 

On the other hand, research tells us that the more practice you have at multitasking and task switching, the better you get, so young folks should be very good at juggling those 10 IM windows by now, minimizing the disruption they cause.  And, just because my research showed that IMs are disruptive to a primary task, that doesn’t mean that they aren’t appreciated and enjoyedJ.  I always have to say that since the finding is often misinterpreted.

 

Let me know if you have any other questions.

Mary

 

From: Tyler Schnoebelen
Sent: Monday, July 24, 2006 3:05 PM
To: Mary Czerwinski
Subject: RE: Some notes on interruption (for users, co-workers, etc)

 

Hi Mary,

 

I came across some of your research as I was looking at interruptibility studies by James Fogarty and co at Carnegie Mellon. In passing, Fogarty mentions your research: “instant message notification is disruptive to task performance even when it is ignored.” Since we’re the folks designing next generation Messenger and Mail services, this is very interesting to us.

 

One of our designers here asked me whether age/familiarity were a factor in how distracted someone was from a task by an incoming IM. What does your experience suggest?

 

Here are the citations for the IM research. I found the papers on Mary’s home page (http://research.microsoft.com/users/marycz/home.htm) and pulled out the subjects’ ages/experience for each citation.

 

Cutrell, E., Czerwinski, M. and Horvitz, E. (2001) Notification, Disruption, and Memory: Effects of Messaging Interruptions on Memory and Performance. Proceedings of the IFIP Conference on Human-Computer Interaction (INTERACT 2001), 263-269.

 

Sixteen participants (nine female) were run through the second study. All were between the ages of 20 and 57, and all but one of the participants had at least tried a chat or instant messaging system before. All were intermediate to advanced computer users. All subjects were run singly for one session.

 

Czerwinski, M., Cutrell, E. and Horvitz, E. (2000) Instant Messaging and Interruptions: Influence of Task Type on Performance. Proceedings of the Australian Conference on Computer-Human Interaction (OZCHI 2000), 356-361.

 

Twelve participants (6 female) aged between 25 and 54 years of age (average age was 41 years old) came to the lab for a single session. All subjects were screened to be at least at an intermediate level of proficiency at using Microsoft Windows and Office products. Six of the subjects had used MSN’s Messenger v. 2.0 prior to the study, and six had never used Messenger before. All subjects were run singly for one session.

 

Czerwinski, M., Cutrell, E. and Horvitz, E. (2000) Instant Messaging: Effects of Relevance and Time. Proceedings of the British HCI Group Annual Conference (HCI 2000), 71-76.

 

14 experienced Microsoft Office 2000 users, aged 26-55, participated in this study.

 

Catchy summary: You probably think your colleagues are more interruptible than they are.

 

Related topics: Working in an open space (or a place with open door policies); interrupting users (alerts for Messenger, Mail, Mobile, etc)

 

This weekend I read a pretty interesting article by James Fogarty and his colleagues at Carnegie Mellon (actually, Fogarty is starting at UW in the fall, I believe).

 

Fogarty et al recorded four managers for 600 hours. At random, twice an hour, these managers were prompted to self-report their level of interruptibility on a five point scale. (These were people who had their doors open 99% of the time, btw.)

 

The researchers took the video tapes and showed them to 40 other people and asked them to assess the level of interruptibility of the person they watched on the video tape. They were given 15 or 30 seconds of video right before the self-reporting event took place. (These “estimators” thought 15 sec was sufficient; the data suggest this, too—showing 30 sec didn’t improve their accuracy any. People assess interruptibility very quickly.)

 

The researchers also hired people to “code” five minutes before each self-report (was the video subject talking on the phone, talking with a person, sitting, standing, etc).

 

Here are the averages for the self-reporting about interruptibility. Not sure how close they might be to what you and I would report:

 

Highly interruptible                                                            Highly non-interruptible

1	2	3	4	5
13.7%	12.8%	24.3%	17.3%	32.0%

 

A guest was present approximately 25% of the time, though there was very rarely more than one guest present.

 

Results:

The estimators only performed slightly better than chance. Importantly, their estimates skewed towards thinking the video subjects were more interruptible than the video subjects thought they themselves were.

 

Overall accuracy was 30.7% (if you just always guessed “highly non-interruptible” you’d be right 29.4% of the time).

 

If you said, wait, let’s just reduce this to a two point scale, “highly non-interruptible” and “everything else”, then people are 76.9% accurate (vs. 70.6% score for semi-chance guessing).

The estimators were confident of the accuracy of their estimates—but higher confidence didn’t make the estimates any better (a little worse, but not in a statistically significant way).

 

The researches then tried to get the coded information to work better than people. The most important factors were “was anyone talking” and “was the video subject on the phone”. Other factors included “are they drinking’, “what time of day is it” (frustratingly they don’t tell us what times of day people are most/least interruptible), “are they using the mouse”, “are they writing”.

 

Different models used different factors, so you need to see the paper to understand all of this fully.

 

Fogarty’s best model yielded 82.4% success for the two point scale, which is (statistically) significantly better than the human estimators.

 

This method was more computationally intensive than other methods that got them 77.8% success (that’s better than the human’s 76.9%, but the difference isn’t statistically significant).

 

The computational models scored 51.5% accuracy for the five point scale, which is a lot better than the human estimators’ 30.7%.

 

The researchers also looked at what sort of things would be “easy to build” and assessed how accurate those would be. 79.2% accuracy is the best they get for their system, better (but not significantly) than the human estimators. Their “easy to build” system is basically a microphone and basic evaluation of keyboard/mouse movements—nothing fancy like a webcam.

 

Caveats:

Perhaps the demographic—managers in a university—is too limited. The sample size is also not tremendously large.

 

The estimators weren’t familiar with the video subjects. It’s possible that they would have performed better if they knew them.

 

For the workplace

 

This research could mean that we need to be more skeptical when we decide that someone is interruptible.

 

In passing, Fogarty mentions other research (3 different articles, actually) that point out that instant message notification is disruptive to task performance even when it is ignored.

 

For user interface design

 

Here are some quotes that present interesting application of interruptibility research in our products:

 

“Because [computer] systems do not have a mechanism for weighing the importance of information against the appropriateness of an interruption, people are forced into extremes of either allowing all interruptions or forbidding all interruptions. This problem is amplified because people forget to re-enable systems after a potentially inappropriate time interval has passed” (2).

 

“Mobile phones could automatically inform a caller that the person being called appears to be busy, allowing the caller to consider the importance of the call in deciding whether to interrupt the apparently busy person or instead leave a message. Email and messaging applications might delay potentially disruptive auditory notifications for less important messages, but never prevent delivery of the information” (2)

 

People do not typically make an initial estimate and then blindly proceed. Instead they look for other cues—is the person I’m trying to interrupt avoiding eye contact, are they continuing the task I’m semi-interrupting? In real life, then, interrupting someone is a ‘negotiated process’: “In designing systems to use interruptibility estimates, it will be important to support a negotiated entry, rather than assuming that interruptibility estimates provides absolute guidance” (12).

 

“Although we had expected talking and the telephone to be important indicators, it is very interesting to note that all 30 of the top individual features are related to either the telephone or talking…if allowed to use only one sensor, a sensor related to talking or the telephone is the most useful” (17).

 

“[Models that use a 5 point scale] provide an additional level of flexibility. People who feel they are being interrupted too often could use the system’s interface to request that they be interrupted less frequently. Instead of initiating a negotiated interruption for a value of 4 or lower, the system could then only negotiate interruptions when its model estimates a value of 3 or lower. Alternatively, systems could use the value of the estimate to decide how subtly to initiate an interruption. Estimates of 3 or 4 could be used by a system to decide to initiate a negotiated interruption with an ambient information display, while estimates of 1 or 2 could be used by the system to decide to initiate with a more direct method” (23-4).

 

You can build systems that learn the individual nuances of people over time (28).

 

“By using a passive approach, instead of requiring people to explicitly indicate interruptibility or create and maintain calendars, our approach makes interruptibility estimation feasible for use in everyday systems. Used with models of the importance of potential interruptions and system designs that support negotiated interruptions, our models offer to support significant advances in human computer interaction” (29).

 

To learn more:

http://www.cs.cmu.edu/~jfogarty/professional.html

 

Fogarty, J., Hudson, S.E, Atkeson, C.G., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J.C., and Yang, J. (2005). Predicting Human Interruptibility with Sensors. ACM Transactions on Computer-Human Interaction (TOCHI), Vol. 12, No.1, March 2005, pp. 119-146.

 

Horvitz, E. and Apacible, J. (2003) Learning and Reasoning about Interruption. Proceedings of the International Conference on Multimodal Interfaces (ICMI 2003), 20-27.

 

Seshadri, S. and Shapira, Z. (2001) Managerial Allocation of Time and Effort: The Effects of Interruptions. Management Science, 47 (5). 647-662.