Thinking Slow: An Argument For Bureaucracy?

Harvard Business School - Harvard University
Published: December 1, 2011

Executive Summary:

Summing Up Readers of Jim Heskett's column this month offer guidelines for when to think fast and when to think slow, from author Daniel Kahneman'sbook, Thinking, Fast and Slow.

About Faculty in this Article:

James Heskett is a Baker Foundation Professor, Emeritus, at Harvard Business School.

• More Working Knowledge from James Heskett
• James Heskett - Faculty
  Research 

Summing Up

Summarizing responses to questions about a topic as complex as thinking slow requires that one think slow. So I wrote this column, slept on it, rewrote it, and still have problems with it. But deadlines, as they so often do, help us avoid the "analysis paralysis" associated with the downside of thinking slow.

So here is what my gut tells me about what you said this month: Thinking slow, as Daniel Kahneman calls it in his recent book by that name, is important under circumstances of high risk, uncertain outcomes, and a limited experience base with the challenge requiring a decision. Good leaders know when to think slow. All of us need mechanisms for helping us to know. Some elements of bureaucracy can be included among the devices. But, as Ganesh Ramakrishnan said, "… we probably (need) to enrich the vocabulary … The word bureaucracy is associated with stagnation, rigidity, unthinking application of age-old rules … (vs.) careful reflective thought on matters that need intelligent application of criteria."

Just how to promote slow thinking where it is appropriate stimulated an interesting and stimulating debate. As Dave Schnedier put it, "I agree with Kahneman's diagnosis, but not his remedy… Whether the decision is to be made by an individual, a team, or 'the bureaucracy', I would say that all would be well served by the discipline … imposed by adherence to a rational process… (which) requires the intellectual rigor that professional management can impose…" Sudheer Thaakur concurred: "… in a world that is more complex and uncertain and ambiguous we should be promoting slow and deliberative thinking. Till we do that we will not be able to fully utilize the opportunity offered by (the) structured nature of organization." Rebecca Mott said "let's not forget the plethora of research that shows teams consistently outperform individuals. It is the modifying effect of the team, which also slows down the decision-making process, that produces a better result." Yadeed Lobo had other suggestions: "Good governance (the objective observer, non executive board members) serves as an effective moderator of (thinking fast) on the part of … executives. The relationship between the Independent Non-Executive Chairman of a Board and the CEO is particularly important."

Some saw a danger in flirting with the use of bureaucratic devices to promote deliberative thought. As Shadreck Salli put it, "What (is critical) … is the person that conducts the switching from thinking (either fast or slow) to (the) other … it may not be easy to have a series of processes in a bureaucratic chain that would best suit a particular type of thinking … Shalia commented: "I think the growth rate of large organizations slow(s) down precisely for this reason-not being able to separate decisions which need deliberations vs. quick decisions." Phillippe Gouambe was more blunt: "… bureaucracy slows everything down and I just do not have the time for that."

As you said, good judgment in thinking fast or slow is an important characteristic of outstanding leadership. Can it be taught or does it have to be acquired over time? How do leaders learn how to think slow at the right time? How do they ensure that others do so as well? What do you think? Or do you need more time?

Original Article

Behavioral economics has fascinated us at least since Daniel Kahneman became the first psychologist to win the Nobel Prize in Economic Sciences in 2002. It has allowed us to see more clearly traditional economics and the fallacy of the assumptions on which it is based: a world of rational humans who are unemotional, well-informed, and take the long-term view in economic decisions.

In contrast, behavioral economics emphasizes and examines the irrational side of managers, consumers, and investors—their tendency to take the short-term view while following their emotions. These "irrationals" are less well informed than they imagine (and certainly less well-informed than traditional economists assume).

As an antidote to these problems, behavioral economists have, for example, advised regulators to combat predictable irrationality in citizens and consumers by "nudging" them by means of economic incentives to act in ways that regulators believe are in citizens' best interests. Lest we underplay the significance of this or somehow associate it with conservative or liberal politics, consider that proponents of such ideas have been advisers to both David Cameron in the U.K. (Richard Thaler) and Barack Obama in the US (Cass Sunstein). Behavioral economists advise regulators that, when in doubt, make sure that business is transparent and even educational in its dealings with others in order to make them wiser.

An observer of policy based on behavioral economics might well conclude that an assumption of irrationality may be valid, but irrational behavior is hard to predict. Why, for example, do people (even experts and managers who should know better) act in their own worst interests or those of their firms? Can such behaviors be predicted and averted?

That is a subject of Daniel Kahneman's new book, Thinking, Fast and Slow. He concludes, based on his research, that as decision-makers, we rely too heavily on System 1 thinking (recall Malcolm Gladwell's book Blink) that helps us summon whatever knowledge we think we have. Among other advantages, it saves us time in situations with similar patterns. But errors of judgment occur when we apply it to complex issues that require more careful consideration, investigation, and reasoning--System 2 thinking. It helps explain behaviors ranging from overconfidence in planning to the defense of sunk costs and the failure to cut our losses in investments.

Kahneman implies, among other things, that in selecting decision-makers we should look for those who know when to switch from System 1 to System 2, that is, when to think fast and when to think slow (using his grammar). Another antidote to thinking fast at the wrong time, according to Kahneman, is to have an objective observer who can flag those situations requiring slow thinking when they arise.

But the best check on too-fast decision-making may be the organization itself. As Kahneman puts it, "Organizations are better than individuals when it comes to avoiding errors, because they naturally think more slowly and have the power to impose orderly procedures." Rather than castigate the bureaucratic nature of large organizations, Kahneman suggests we try to make them more efficient at slow thinking by improving processes for deliberation and making decisions

.

Do you agree with what Kahneman implies here? Does bureaucracy have its advantages? Is this one of them? What do you think?

To read more:

Malcolm Gladwell, blink: The Power of Thinking Without Thinking (New York: Little, Brown and Company, 2005)

Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and Giroux, 2011)

Richard H. Thaler and Cass R. Sunstein, Nudge: Improving Decisions About Health, Wealth, and Happiness (London: Penguin Books, 2008)

Jim Heskett's latest book,The Culture Cycle, was published in September. 

China Inc. Goes Global

Columbia Business School - Columbia University
Published: December 12, 2011

China’s economy is taking its next great leap forward: parts of its manufacturing sector are now moving up the value-added chain and out of the country. The China challenge is now a global one.

The reasons are not difficult to fathom. Production costs (wages, office rents, land, capital, etc.) in China’s coastal provinces — where most of the country’s manufacturing and service production, as well as foreign direct investment, are located — have been rising fast. Since last year alone, minimum wages in 9 of 12 coastal provinces (including Beijing) rose by an average of more than 21 percent.

At the same time, the renminbi is appreciating, making domestic production of export-oriented goods and services even more expensive. This matters, especially for labor-intensive activities (ranging from toy manufacturing to data-entry services), whether by affiliates of foreign multinational enterprises (which account for more than half of China’s exports) or by local firms, which are losing competitiveness in international markets.

To maintain its export-oriented production base, output must move up the value-added chain, toward more sophisticated products. Multinational enterprises can do that within their integrated global production networks, which allow them to organize an international intra-firm division of labor. Any part of these production chains can be located wherever it suits the firms’ international competitiveness best. And such firms have the experience to scout the globe for the right investment locations.

Domestic Chinese firms, too, need to respond to these pressures. They are helped in this effort by the rapid deepening of China’s skills and technology base. This partly reflects training in foreign affiliates, but the main reason has been the Chinese government’s sustained effort to foster education and training, encourage technology transfers from foreign to domestic firms, and, in particular, to build up research and development capabilities.

In short, producers of more sophisticated goods and services in developed and emerging-market countries need to be prepared for growing competition from China.

The Shifting Labor Landscape

At the same time, China’s labor-intensive production will increasingly move to countries with lower labor costs — including Bangladesh, India, Indonesia, and neighboring Vietnam (where Chinese firms have already established about 1,000 affiliates), as well as various African countries. This process has already begun and has been supported since the beginning of the last decade by the government’s “going global” policy, through which it encourages outward foreign direct investment (FDI) from China.

The data bear this out: FDI outflows more than doubled in 2008, to $52 billion, from $23 billion in 2007, and rose even further in 2009 (when world FDI flows collapsed by about 50 percent, owing to the Western financial and economic crisis), before reaching $68 billion in 2010. Not counting Hong Kong, this made China the world’s fifth largest outward investor that year.

This development creates new opportunities for other emerging economies to reap the trade benefits of inserting themselves into the international division of labor. These countries’ investment-promotion agencies — indeed, those of all countries, including developed ones — should increasingly target firms in China to lure them to their shores. In so doing, they should aim not only for big state-owned companies but also for the rising number of vibrant, private small and medium-size enterprises in China that can be found in all sectors of the economy.

But an important caveat is in order: China has a vast interior that is far less developed than the coastal provinces. The government is making special efforts to develop these areas in the framework of its western development strategy, including building modern infrastructure, promoting high-quality education, supporting science and technology (all key determinants of the location of production), and encouraging investment there. As a result, firms based in the coastal provinces that have to move their production (and see no need to diversify away from China) can choose to relocate to China’s interior rather than going abroad.

What Governments Need to Do Now

The pattern is clear: this sort of transition away from labor-intensive manufacturing happened before in today’s developed countries, when firms headquartered in Europe, Japan, and the United States moved production to developing countries. Asia, Hong Kong, South Korea, Singapore, and Taiwan were (and have been) among the beneficiaries.

When costs for labor-intensive goods and services became too high in these countries, production was shifted elsewhere. This relocation of manufacturing has since been accompanied by the offshoring of services whose information-intensive components have become tradeable.

China itself has benefited from today’s open international trade and investment regime, which allows firms to locate production where it is most beneficial for their international competitiveness — and is now beginning to shed labor-intensive industries itself.

Governments need policies to adapt to this global shift in production. They should help their countries’ firms to adapt to the departure of some producers by establishing training programs, stimulating innovation, and maintaining or creating a competitive environment that encourages “creative destruction” while providing for a social safety net.

Likewise, governments that attract the production that was shed elsewhere need to have policies in place that enable them to benefit as much as possible from this global shift, thereby furthering their own economic development.

Karl P. Sauvant is executive director of the Vale Columbia Center on Sustainable International Investment at Columbia University.

Copyright: Project Syndicate, 2011

Are Creative People More Dishonest?

Harvard Business School - Harvard University
Published: December 7, 2011

In his 1641 treatise, Meditations on First Philosophy, philosopher René Descartes introduced the concept of an "evil genius," a powerful force of nature who is equally clever and deceitful. Since then, the world has given us plenty of examples—Hannibal Lecter in The Silence of the Lambs, fictional Wall Street villain Gordon Gekko, and real-life Wall Street villain Bernie Madoff, to name a few. Not only were these classic bad guys unquestionably unethical, but all were inarguably creative in carrying out their bad behavior as well. Indeed, it's rare to hear anyone described as both evil and unoriginal.

This raises a question: Is there a link between creativity and unethical behavior?

"Dan and I started wondering whether there is something about the creative process that triggers dishonest behavior."

There certainly is, according to an article in a forthcoming issue of the Journal of Personality and Social Psychology. In "The Dark Side of Creativity: Original Thinkers Can Be More Dishonest," the authors report that inherently creative people tend to cheat more than noncreative types. Furthermore, they show that inducing creative behavior tends to induce unethical behavior.

It's a sobering thought in a corporate culture that champions out-of-the-box thinking.

"In any organization, especially in contexts that are global and very competitive, there is so much focus on trying to be innovative and creative," says Francesca Gino, an associate professor at Harvard Business School, who wrote the article with Dan Ariely of Duke University. "But is creativity always good? We often hear of cases in which people use innovative behavior to create a sense that what they're doing is not morally wrong. So, Dan and I started wondering whether there is something about the creative process that triggers dishonest behavior. Specifically, we decided to explore the idea that enhancing the motivation to think outside the box can drive individuals toward more dishonest decisions when facing ethical dilemmas."

Creativity and ambiguity

To begin their research, Gino and Ariely surveyed 99 employees across 17 departments at an American advertising agency, where some jobs—copywriting, for example—required much more creativity than others. In the anonymous survey, on a seven-point scale, the respondents indicated how likely they were to engage in various ethically questionable work behaviors such as "take home office supplies from work" and "inflate your business expense report." Respondents also evaluated scenarios describing a hypothetical person who has the opportunity to behave dishonestly, and then indicated, again on a seven-point scale, how likely they would be to behave unethically in each instance. Finally, the respondents reported how much creativity was required in their respective jobs, with three managers in the executive office rating the creativity level required in each department, as well.

Overall, the researchers learned, the higher the creativity required for the job, the higher the level of self-reported dishonesty.

Then, through a series of experimental studies, the researchers tested--and largely proved--the theory that creative people are more likely to exhibit unethical behavior when faced with ethical dilemmas.

The first study tested the hypothesis that a naturally creative person is predisposed to dishonest behavior. (The week before the experiment, the participants, 71 university students, completed an online survey that included dispositional measures of creativity.) The experiment included a computerized task in which participants viewed 20 dots inside a diagonally bisected square. They were told to indicate whether there were more dots on the right side of the square or on the left, and that their answers would affect how well they would be compensated for taking part in the experiment: each "more-on-the-right" decision would earn them 10 times as much as a "left" decision.

In half the trials, it was obvious that one side of the square had more dots than the other—2 dots versus 18, for example. But in the other half, the task was a little more ambiguous, with several dots appearing near or on the line in the middle of the square. The researchers focused on the results of the "ambiguous" tasks, with the idea that these were the ones that allowed more room for interpretation—participants could easily misrepresent what they actually perceived and report "more on the right" in order to incur a higher payoff.

The results showed that participants who had scored high on the creativity scale were the most likely to fudge their answers for monetary gain.

"Ambiguity, having some room to justify our behavior, seems to be a really important component of explaining when and why we cross ethical boundaries, and these results show us that creativity helps with that process," Gino says. "It suggests that moral flexibility is the mechanism explaining why being in a creative mindset or being a creative person puts you more at risk to do the wrong thing."

The perils of inducing creativity

In another study, which included 111 university students, the researchers tested whether they could actively induce creativity, and whether doing so would temporarily induce dishonest behavior. Participants were randomly assigned to one of two groups: the "creative mindset" group and the control group. All were asked to construct sentences from sets of randomly positioned words. But in the creative mindset group, more than half of the sentences included words related to creativity: "novel," "imagination," "invention," "originality," and so on.

"We're not saying that creativity's bad, but we are saying that it can lead to problems."

To test whether the creativity prime worked, the researchers asked participants to solve a cognitive puzzle created by the Gestalt psychologist Karl Duncker. Known as Duncker's candle problem, it presents participants with the task of affixing a candle to a wall in such a way that when lit, the candle won't drip wax on the floor. To complete the task, participants can use a box of tacks, a book of matches, and the candle. The ideal solution, which requires ingenuity, involves emptying the box, tacking the box to the wall as a candleholder, placing the candle inside, and lighting the candle with the match. The researchers found that 47.3 percent of participants in the creative mindset group solved the candle problem ideally, versus 26.8 percent in the control group.

Next, participants completed a series of computerized tasks, including the ambiguous dots-in-the-square task from the first study. The results showed that those in the creative mindset group were much more likely to give dishonest answers for monetary gain than those in the control group.

"These were simple studies, but they were powerful in showing that our ability to justify things is significantly greater if we are in a creative mindset or when we are creative people," Gino says.

That said, Gino is quick to add that she and Ariely are not suggesting that companies put the kibosh on innovation in order to keep dishonesty at bay.

"We're not saying that creativity is bad," Gino says. "But we are saying that it can lead to problems. And so the question from a manager's perspective is: How do you get the good outcomes of creativity without triggering the bad outcomes?"

While "The Dark Side of Creativity" doesn't answer that question directly, Gino hopes that the research will remind innovative organizations not to give short shrift to ethics.

"As a manager, if you're highlighting the importance of being creative and innovative, it's important to make sure that you're stressing the presence of ethics, too," Gino says. "Dan and I are of the hope that managers will start thinking about how to structure the creative process in such a way that they can keep ethics in check, triggering the good behavior without triggering the bad behavior."

The Most Powerful Workplace Motivator

Harvard Business School - Harvard University
Published: October 31, 2011

Any parent can tell you that a surefire way to turn joy into rage is to offer your child a big candy bar—and then turn around and offer an even bigger one to his sister. Suddenly, a special treat turns into a great injustice. "Hey! How come she got more? That's not fair!"

And any hiring manager can tell you that the world of business is not so different.

"It really was all about the recognition of and comparison with their peers, and many of them were willing to pay for it."

"This is why MBA programs send out lists of average salaries, and why students spend hours poring over those lists," says Ian Larkin, an assistant professor in the Negotiation, Organizations & Markets Unit at Harvard Business School. "You should see the angry e-mails I get from students when they find out that a job offer turns out to be $10,000 per year below the average. It's not that they really feel like an annual salary offer of $115,000 is unfair on its own. They might be perfectly happy with that salary if it weren't for the information that it's below average."

And it's not just a matter of money. In several studies of social comparison in the workplace, Larkin has found that the most powerful workplace motivator is our natural tendency to measure our own performance against the performance of others.

"Traditionally, [the field of] economics has held a very rational view of people, and there's a gigantic amount of literature focusing on financial incentives and the idea that simply having financial incentives causes people to work harder," he says. "But my research suggests that in deciding how hard we work and how well we think we're performing, social comparisons matter just as much."

The $30,000 gold star

The power of social comparison can lead to irrational financial decisions, according to Larkin's 2009 paper "Paying $30,000 for a Gold Star: An Empirical Investigation into the Value of Peer Recognition to Software Salespeople."

The paper describes a field study at a large enterprise software firm, where salespeople's salaries are largely based on commissions. The firm also features another common sales incentive--a "president's club" membership for those employees who sell more software than 90 percent of their peers in a given year.

The software firm uses a "commission accelerator" program over the course of each financial quarter, meaning that a salesperson expecting a high-volume sale at the beginning of a quarter would receive a higher commission on any additional sales in the same quarter. A salesperson expecting a large sale early in the first quarter of the year would rationally want to delay any other potential sales until later in that quarter, so as to take advantage of the accelerating commission schedule.

However, making the sale right away, before the end of the year, could help the salesperson achieve special recognition as a member of the club. Thus, the salesperson faces a choice: delay the sale and garner eventual commission boosts, or make the sale right away and improve the chance of attaining club membership. In the paper, Larkin uses actual choices of hundreds of salespeople facing this decision to statistically estimate the average salesperson's "willingness to pay" for club induction—the point at which a salesperson is indifferent to waiting for greater commissions and closing the deal now and getting inducted into the club. The willingness-to-pay statistic at the software firm is calculated to be nearly $30,000, or approximately 5 percent of take-home pay.

"My research shows that salespeople who are right on the margin of club induction are actually willing to pay to get over the margin and into the club," Larkin says.

Importantly, Larkin observed that there were no apparent financial benefits to attaining club membership. Recipients received a gold star on their name card, companywide recognition, an e-mail from the CEO, and a weekend trip to a tropical destination with the other club members. (Granted, the trip was worth several hundred dollars, but was far less financially valuable than a large commission.)

Club members "were not more likely to be promoted, leave for a better job, or make higher commissions in the future," Larkin says. "It really was all about the recognition of and comparison with their peers, and many of them were willing to pay for it."

Insecurity leads to dishonesty

Social comparison also can lead to insecurity-driven cheating, as Larkin details in a 2009 paper co-written with HBS colleague Benjamin Edelman, Demographics, Career Concerns or Social Comparison: Who Games SSRN Download Counts? The paper addresses an issue near and dear to academics worldwide: the relative popularity of working papers in the Social Science Research Network (SSRN) repository.

The SSRN is a huge academic paper repository, with more than 100,000 authors and 500,000 registered users who have the opportunity to view or download every paper on the site. For each paper, SSRN creates a web page that includes statistics on how many times the paper has been downloaded and viewed. The SSRN site also publishes various "top 10" lists in numerous fields, ranked according to how many times the paper has been viewed, downloaded, or cited elsewhere on SSRN.

Some scholars paid a lot of attention to the reported download counts of their papers; Larkin reports that one prominent legal academic described the monitoring of his own paper's download counts as "like crack for me."

Historically, SSRN allowed unlimited downloads of papers, and most of those downloads were reflected in the reported download count on each paper's web page. It became apparent that many authors were gaming the download count system by repeatedly downloading their own papers, so that others would see the high download count and assume that these particular papers were very popular. SSRN maintains detailed historical records of every paper download and is able to determine when papers appear to be downloaded over and over by the same person.

"It's like having a convenience store that's not manned, and everyone who comes in can either steal or pay, but there's a video camera that nobody knows about, and it's tracking everyone's every move," Larkin says. "For years, some academics got away with inflating their own download count numbers, but we were able to see exactly who was doing this, and in what circumstances."

In their research, Larkin and Edelman teamed up with the SSRN and set out to determine the factors that would make academics inflate the download counts of their own papers.

"As economists, we thought, hmmm, it's probably people who are up for tenure soon, or maybe it's the people who just graduated, and they want to get their name out there," Larkin says. "We were thinking very much along the traditional economic model--people doing things for rational, career-promoting reasons."

Larkin shared these hypotheses with HBS colleague and mentor Max Bazerman, a leading ethics scholar, who had a different theory. "Max told me, 'I'll bet people are doing this because they feel bad that their papers aren't being downloaded as much as their colleagues' papers,' " Larkin says. "So we looked at that."

It turned out Bazerman was right. The researchers found that authors were more likely to download their own papers repeatedly when a colleague's paper was performing especially well on the site, or when a very similar paper to an author's was newly released and received significant downloads. Deceptive downloads also increased during times when a paper was close to gaining (or losing) placement on a top 10 list. (Ironically, one of the most downloaded SSRN papers of all time is 'I've Got Nothing to Hide' and Other Misunderstandings of Privacy.)

"Again, what was surprising to us was how little we found in terms of the economic reasons for doing this," Larkin says. "By far, the biggest predictor of this behavior was fear of being socially inferior to one's peers."

(Those tempted to boost a paper's usage stats should note that SSRN's terms of service now state that the attempted manipulation of download counts is against site rules, and that the organization retains the right to ban anyone caught abusing the system.)

Ramifications for salary managers

The field evidence from the worlds of software sales and academia indicates that companies need to bear social comparison in mind when designing compensation plans. Larkin discusses the issue in The Psychological Costs of Pay-for-Performance: Implications for the Strategic Compensation of Employees, a paper he cowrote with HBS colleague Francesca Gino and Washington University's Lamar Pierce.

The authors argue that paying each employee solely according to his or her performance is actually an inefficient strategy; it can lead to resentment or even sabotage on the part of employees who believe they are underpaid compared with their colleagues. Thus, a standardized salary scale, combined with ancillary incentive programs, may be the best way to motivate employees. "When deciding how much effort to exude, workers not only respond to their own compensation, but also respond to pay relative to their peers as they socially compare," the paper states.

That's important food for thought, considering that Facebook, LinkedIn, and other such sites have made it de rigueur to share information that we used to keep to ourselves.

"It used to be that our salaries were very secret, but they're getting less and less secret because of social networking," Larkin says. "And people get upset quickly when they realize that there are large variances in how much other people are paid. Companies need to realize that with the overflow of information these days, paying peers differently is going to affect not only how those people feel but how their colleagues feel as well." 

What Makes A Good Leader?

Harvard Business School - Harvard University
Published: April 2, 2001

When discussing business leadership, the distinction between good management and good leadership is often made. Managers are thought to be the budgeters, the organizers, the controllers — the ants, as one observer puts it — while leaders are the charismatic, big-picture visionaries, the ones who change the whole ant farm. But such a construction, those interviewed for this article agree, erroneously leads to a bimodal way of looking at something that should really be evaluated on two separate scales. "Everybody has got a little bit of each in them," says John Kotter, who admits he is sometimes guilty of using the dichotomy in an effort at simplification. "It's much better to think in terms of measuring people on a zero-to-ten scale for each quality."

HBS professor Joe Badaracco agrees that the traditional manager versus leader argument ("Clark Kent versus Superman," he jokes) tends to undermine the value of management. "There are lots of people who look and act like managers, who have excellent managerial skills, and who don't make a lot of noise. Nobody is writing cover stories about them. But after they have been in an organization for a period of time, things are significantly better," observes Badaracco. "Now, are these mere managers because we can't compare them with Martin Luther King? Or are they leaders because they accomplished something that needed to be done?"

If leaders disclosed all their concerns and doubts, stock prices would plummet, their competitors would be all over them, and employees would be jumping ship.
—HBS professor Joe Badaracco

Some great managers struggle with change and fail to be great leaders, while a great leader might fail to create a sense of stability in an organization and not measure up as a manager. HBS professor David Thomaspoints out that "increasingly, the people who are the most effective are those who essentially are both managers and leaders."

Communication Is Key

"Communication is the real work of leadership," says HBS professor Nitin Nohria, who documented the importance of persuasion in his 1992 book Beyond the Hype: Rediscovering the Essence of Management. Nohria believes effective leaders are masters of the classical elements of rhetoric, as outlined by Aristotle centuries ago. "You can reach people through logos or logic, by appealing to their sense of what is rational," he explains. "You can use pathos, appealing to their emotions, or you can make an argument based on their sense of values or ethos." Great leaders, he notes, "spend the bulk of their time communicating, and they know how to employ all three of Aristotle's rhetorical elements."

Nohria also feels that leaders are able to distill their message, however complex it may be, to something that is accessible to those who may not share their knowledge or background. Joe Badaracco agrees. "You need a talent for simplicity — for saying things in a few words. General Electric's Jack Welch is a good example. He is astonishingly articulate and able to convey complicated concepts in just a few phrases."

Of course, knowing your audience is also essential. "Great communicators have an appreciation for positioning," states John Kotter. "They understand the people they're trying to reach and what they can and can't hear. They send their message in through an open door rather than trying to push it through a wall." Badaracco believes part of knowing your audience is the ability to listen. "Communication can't always follow the top-down model," he says. "With the fluidity of information in business today, leaders need to be masterful listeners; they need to be able to receive as well as send."

David Thomas stresses the importance of "multimodality" in communication. "What you say is only the beginning," he states. "Your behavior, your actions, and your decisions are also ways of communicating, and leaders have to learn how to create a consistent message through all of these. It's been said many times, but leaders lead by example."

For Rosabeth Moss Kanter, a key question is whether a leader's personal passion matches his or her aspirations. "There are so many false starts, unexpected obstacles, and surprising turns along the path to change. Daily work often drains energy needed for change," she says. "Leaders must pick causes they won't abandon easily, remain committed despite setbacks, and communicate their big ideas over and over again in every encounter."

Telling the Hard Truths

What happens when leaders must communicate facts that are hard to take? Nitin Nohria reflects on Winston Churchill's devastating defeat at Gallipoli, which resulted in over 100,000 Allied casualties during World War I. "The campaign was a total fiasco for British military leadership," he notes. "When it was over, Churchill took complete responsibility. A setback like that could have been paralyzing, but he was able to move forward to lead his country to victory in World War II."

The lesson, says Nohria, is that Churchill and other great leaders are pragmatists who can deal with difficult realities but still have the optimism and courage to act. "Enduring setbacks while maintaining the ability to show others the way to go forward is a true test of leadership," he asserts.

Jeff Bezos, founder and CEO of Amazon.com, has said that one of the key elements of being a good business leader is the capacity to tell the hard truths. "Leaders struggle with this problem all the time," says David Thomas. "From a leadership point of view, you always want to move toward telling the hard truths and helping people cope with the realities of change. But as a manager, you might be more inclined to minimize the complexity of a situation so things can run smoothly for as long as possible. It's often a judgment call."

The ability to render that judgment can sometimes make or break a company. "The phrase 'public confidence, private doubt' comes to mind," observes Joe Badaracco. "If leaders disclosed all their concerns and doubts, stock prices would plummet, their competitors would be all over them, and employees would be jumping ship. But even if you can't be absolutely open with everyone, leaders have to confront their companies' problems and, of course, share them with top management."

John Kotter underscores the positive potential of facing problems head-on. "Great leadership does not mean running away from reality," he argues. "Sometimes the hard truths might just demoralize the company, but at other times sharing difficulties can inspire people to take action that will make the situation better." 

Ultimate Question In Management

Harvar Business School - Harvard University
Published: November 3, 2011

Is the Ultimate Question, Why?

What might be called the "ultimate question exercise" this month yielded a number of interesting responses. First, nearly everyone was willing to play the game in suggesting their own favorite ultimate questions for managers. Nominees took on several forms. But many proposed questions were clustered around inquiries about willingness to recommend, trust, and authenticity.

Among suggested questions that managers might ask of themselves were: "What positive, relevant impact can I make?" (from Clinton Coker), "How do you continuously inspire the passion and confidence for every employee to perform their role at the expected level or better?" (from Barry Cohen), "Will the world be a better place if I do my job well?" (from Mo Bjornestad), "Do you want this position so you can serve—or so you can be served?" (from David Witt), "What are you doing to enhance (your) … company's, (your) … team's, (your) … organization's development"? (constructed from comments by Marlis Krichewsky), and "What are you doing for others?" (from Dennis Hopwood). One of the more interesting questions of this type was proposed by Pete DeLisi: "Do you make your people feel like they want to be a better man or woman?"

Favorite "ultimate" questions that managers could ask of their reports included several adherents to "Do you trust your boss?" However, Peter Bowie observed that "fundamental to trust is integrity so I would build the foundation on integrity." Other favorites included "the extent to which I feel my manager or my organization is being real with me (… doing what it says on the tin)" (from Jackie Le Fevre), "Does our value system determine the behavior of management or does the behavior of management determine our value system? (from Athan Sunderland), "Is (our) … organization a place where balanced risks can be taken without causing too much career retardation?" (from Yadeed Lobo), and "Is my manager characterized by authenticity and passion?" (implied by a response from Harry Abrikian). Steve Sheinkopf would alter the process to insure that he receives feedback only from his "A players, not by the Bs (or Cs)."

The coauthors of The Ultimate Question 2.0 weighed in as well with some background on their choice of "How likely is it you would recommend us to a friend?" as the ultimate question. Fred Reichheld put it this way in describing the question behind the question: "our candidate for the ultimate question in business (and in life) is: Have we treated others … with dignity and honor—in a manner consistent with the Golden Rule?" And Rob Markey pointed out that "Trust does lie at the heart of someone's likelihood to recommend …" He went on to say that "we have always advocated pairing the likelihood to recommend question with what is perhaps even more important: 'Why?'" So is the ultimate question really "Why?"? What do you think?

Original Article

The publication this month of The Ultimate Question 2.0 (revised from an earlier edition) provides us with an opportunity to ask ourselves just what is the ultimate question in management.

In their book, Fred Reichheld and Rob Markey remind us of the simplicity of the Net Promoter Score (NPS). It's the product of answers to one question, "How likely is it you would recommend us to a friend?" The NPS has become so popular that, as a customer, you quite likely have been asked that question in the past couple of months. Those replying with a 9 or 10 (the most positive) on an 11-point scale (0 to 10) are "promoters"; a 7 or 8 labels you as a "passive"; and anything from a 0 to a 6 makes you a "detractor." Subtract the proportion of detractors from the proportion of promoters and you get a "net promoter score" that can range anywhere from +100 to -100.

And that's it. Tracking the net promoter score, according to the authors, can lead to improvements in both management and performance.

As managers and students of management, we have a tendency to want to simplify things. Evidence of this is the plethora of management books with single word titles such as Accountability, Transparency, and Teamwork. We search for the one key to management success. Based on recent research, I have my own candidate for that "one key thing:" trust. (There's precious little trust in government, Wall Street, and business in general these days.) I found a strong correlation between trust, loyalty, engagement, and "ownership" among employees in a sample of organizations I examined. Respondents in the study made a convincing case that trust was absolutely essential to the successful implementation of policies and practices necessary to implement any strategy. For example, several managers testified to the importance of the relationship between trust and the ability to achieve speed in getting things done. It's a topic that Stephen M. R. Covey wrote persuasively about several years ago in his book, The Speed of Trust. So for me one candidate "ultimate question" would be "Do you trust your manager?" or "Do you trust your organization?"

My study led to an exploration of the underpinnings of trust, as suggested by related survey data. One major determinant is whether a manager or the organization does what it says it will do, whether it lives up to "the deal" on things important to an employee, whether it meets that employee's expectations. So another "ultimate question" might well be "Does your manager do what she says she will do?" or "Does your organization do what it says it will do?"

What is the ultimate question in management? Or do you object to playing this game?--The Net Promoter Score certainly has its detractors. All of these are efforts to provide simple guideposts in a very complex process. Performance measurement can be a confusing process, leading to inaction or, worse yet, inappropriate action. Can an "ultimate question" have a useful management function? If so, what's yours? What do you think?

To read more:

Stephen M. R. Covey with Rebecca R. Merrill, The Speed of Trust: The One Thing That Changes Everything (New York: Free Press, 2006).

Fred Reichheld and Rob Markey, The Ultimate Question 2.0 (Revised and Expanded Edition): How Net Promoter Companies Thrive in a Customer- Driven World (Boston: Harvard Business Press, 2011).

Jim Heskett's latest book,The Culture Cycle, was published in September. 

Walking Through Doorways Causes Forgetting

The Quarterly Journal Of Experimental Psychology
Published: May 10, 2011

Abstract

Jump to section

• EXPERIMENT 1
• EXPERIMENT 2
• EXPERIMENT 3
• GENERAL DISCUSSION

Previous research using virtual environments has revealed a location-updating effect in which there is a decline in memory when people move from one location to another. Here we assess whether this effect reflects the influence of the experienced context, in terms of the degree of immersion of a person in an environment, as suggested by some work in spatial cognition, or by a shift in context. In Experiment 1, the degree of immersion was reduced by using smaller displays. In comparison, in Experiment 2 an actual, rather than a virtual, environment was used, to maximize immersion. Location-updating effects were observed under both of these conditions. In Experiment 3, the original encoding context was reinstated by having a person return to the original room in which objects were first encoded. However, inconsistent with an encoding specificity account, memory did not improve by reinstating this context. Finally, we did a further analysis of the results of this and previous experiments to assess the differential influence of foregrounding and retrieval interference. Overall, these data are interpreted in terms of the event horizon model of event cognition and memory.

Keywords

• Event models, 
• Situation models, 
• Memory

Work on event cognition has revealed a location-updating effect, which is the finding that when people pass through a doorway to move from one location to another, they forget more information than if they do not make such a shift (Radvansky & Copeland,2006; Radvansky, Tamplin, & Krawietz, 2010). In this work, the environments people moved through were virtual ones. This effect is in line with other research in text comprehension that shows that memory declines when there has been a shift in location (e.g., Curiel & Radvansky, 2002; Morrow, Greenspan, & Bower, 1987; Radvansky & Copeland, 2010; Radvansky, Copeland, & Zwaan, 2003; Rinck & Bower, 1995). Essentially, a shift at an event boundary introduces a need to update one's understanding of the ongoing events, and this updating process is effortful. Also, this finding is consistent with work on event segmentation theory (Kurby & Zacks, 2008; Swallow, Zacks, & Abrams, 2009; Zacks, Speer, & Reynolds, 2009). Specifically, as people parse events, information that was present prior to an event boundary, such as a shift in location, becomes less available after the shift. The aim of the current study is to explore whether this finding is dependent on how the environments are experienced.

In Radvansky and Copeland's (2006) original work, people progressed through a multiroom virtual environment using a large 66″ diagonal display screen, with people sitting about 1 metre away, to provide a high degree of immersion. Each room had one or two tables. A person walked towards the table, set down one object (coloured solids, such as a red cube or a blue wedge), picked up another, walked to the next table, which was either across the large room or in another room, and so on. The critical comparison was whether there was a shift to a new room or not. Note that the distance travelled was held constant, regardless of room change.

At critical points, people were probed with the name of an object (e.g., red cube) either halfway across a large room (no-shift condition) or just after having entered a new room (shift condition). Following Glenberg, Meyer, and Lindem (1987), positive responses were to be made if the probe was either the object that was being carried or the one that was just set down. Negative responses were to be made to probes that were recombined object and colour names from the two positive objects. So, if the object set down was a yellow cone, and the carried object was a blue wedge, a negative probe could be either “yellow wedge” or “blue cone”. Note that the objects could not be seen when probed. The carried object disappeared when it was picked up, and people turned “their backs” from the object that had been set down. Moreover, the probes did not occur at every possible location, which decreased the degree to which people could anticipate them.

The results showed a location-updating effect in which people made more errors if they had moved to a new room. This accuracy difference was also supported by a response time difference, with people responding slower to probes in the shift than in the no-shift condition. Thus, event model updating compromised memory.

Recently we have been developing the event horizon model of event cognition and memory. This model has five components: (a) Events can be segmented, and different event models are created with people processing one at a time; (b) information in the current event that is being actively processed is foregrounded; (c) there is retrieval facilitation for noncompetitive retrieval; (d) there is retrieval interference for competitive retrieval; and (e) there is the storage of causal relations among events. Aspects of this model can be drawn upon to explain the location-updating effect. Of particular concern here are the first, second, and fourth components.

First, event segmentation occurs when an event boundary is encountered, such as a person moving from one room to another, and a new event model may be created and stored in memory (e.g., Kurby & Zacks, 2008; Swallow et al., 2009; Zacks et al.,2009). The event model for the prior event then declines in availability. Note that, for simplicity, we assume that there can only be one event model active at one time. Once an event model becomes deactivated, it will decline in availability until it reaches some background level of activation.

Second, the event model that is currently active in working memory is foregrounded, and it is easier to retrieve information from that event (e.g., Glenberg et al., 1987; Zwaan, 1996). This is to say that the current event model occupies working memory, and available processing capacity is directed to it.

Finally, and particularly important in these experiments, memory for the objects was assessed using a recognition probe. Thus, a person must choose one memory trace to verify that information. When people move an object from one location to another, it is now associated with two locations—the one that it was picked up in, and the one where it was carried. Thus, there may be two event models that contain the target information, which compete with one another at retrieval, producing interference, and making retrieval slower and more error prone (e.g., Radvansky, 1999). In essence, this is a kind of a fan effect (Anderson,1974), and the more locations an object is in, the harder it is to retrieve one particular event model (Radvansky, 1998, 1999,2005, 2009; Radvansky & Copeland, 2006; Radvansky, Spieler, & Zacks, 1993; Radvansky & Zacks, 1991), even though in this case they would all support the same response.

An alternative account of the location-updating effect is that it is an artefact of the degree of immersion a person is experiencing while navigating the environment. First, it may be that this effect requires a higher degree of immersion. This has been referred to the degree of presence that is derived from a mediated experience (Lombard & Ditton, 1997). For example, a radio broadcast or a book would provide lower degrees of presence than a television show or a movie. Moreover, virtual-reality technologies would provide an even higher degree of presence. This would be consistent with some research on mental maps showing differences between memory after learning a layout of an environment while being immersed in it or learning it from a map (e.g., McNamara, Altarriba, Bendele, Johnson, & Clayton, 1989), although it should be noted that this work involved different perspectives as well as different degrees of immersion. From this view, a greater sense of immersion provides a richer representation of the environment, thereby allowing cognition to be more influenced by structural characteristics. Experiment 1 addressed whether decreasing the degree of immersion, by using standard computer monitors, would impact the presence of a location-updating effect.

Alternatively, it may be that virtual environments, while more immersive than those experienced with standard computer monitors, are not complete substitutes for reality. As such, the virtual environment may still be distinct from processing event information in a real situation (e.g., McNamara et al., 1989). As such, in Experiment 2, we increased the degree of immersion by making the situation maximally immersive by testing for a location-updating effect in actual reality.

Finally, Experiment 3 addressed an alternative interpretation of the location-updating effect. Previous research has shown that environmental factors affect memory. The encoding specificity phenomenon posits that information learned in one environment is retrieved better when retrieval occurs in the same context (Thomson & Tulving, 1970). We tested whether the location-updating effect was due to the updating of event models (i.e., the number of rooms entered) or the encoding specificity phenomenon (i.e., whether encoding and retrieval occurred in the same context).

EXPERIMENT 1

Jump to section

• EXPERIMENT 1
• EXPERIMENT 2
• EXPERIMENT 3
• GENERAL DISCUSSION

The aim of Experiment 1 was to assess what sort of impact reducing the degree of immersion would have on the location-updating effect. If the effect requires a higher degree of immersion, perhaps because event updating requires that a person experience the event more directly as a part of the event and the concomitant influences of structural characteristics environment, then reducing screen size would reduce the feeling of immersion and the location-updating effect. Alternatively, if the location-updating effect is due to the tracking of information across events, regardless of whether one is embedded in that environment, then the location-updating effect would still be evident. In Experiment 1, we reduced the degree of immersion by using standard 17″ diagonal monitors rather than a 66″ diagonal screen. With the smaller monitors, the virtual environment does not fill the visual field as much as the larger monitors. Larger displays allow for greater immersion, leaving the impression of being contained and present within the virtual world (e.g., Bystrom, Barfield, & Hendrix, 1999).

Method

Participants

Fifty-five people (31 female) were recruited from the University of Notre Dame participant pool and were given partial course credit for their participation.

Materials and apparatus

The virtual spaces were created using the Valve Hammer program (Vale Software, 2003), which is used to create environments for the Half-Life video game. For this experiment the displays were standard 17″ diagonal monitors. The virtual space was a 55-room environment that had rooms that were two possible sizes, with larger rooms being twice as long as small rooms. This difference in room size allowed for the distance travelled to be equated in the shift and no-shift conditions. Included in each room were one or two rectangular tables. Each table was placed along a wall. For the small rooms there was only a single table, whereas for the larger rooms there was a table in each half of the room. At one end of the table was the object to be picked up. The other half of the table was empty for the object carried from the previous room to be set down. Additionally, the two doorways in a room were never on the same wall so participants did not repeat any part of the path.

The objects were combinations of colours and shapes. The colours used were: red, orange, yellow, green, blue, purple, white, grey, brown, and black. The shapes were: cube, wedge, pole, disc, cross (X), and cone. All combinations of colours and shapes were possible, and they were all used, although some were not probed for. A given shape–colour combination did not repeat as an object in the experiment. People moved through the virtual environment using the left hand to press the arrow keys on the computer keyboard.

Procedure

After signing an informed consent form, people were seated about 0.67 metres from the display. They were told that the task was to pick up an object from the table, move to the next one by either walking across a large room (no shift) or passing through a doorway to the next room (shift), place the object on the empty part of the table, pick up the next object, and so forth. Picking up and putting down objects was done by touching the appropriate end of the table.

To ensure that people progressed through the rooms in the required order, after a person had entered a room, the door behind the person closed. The door to the next room did not open until the person put the object they were carrying on the table and picked up the new object.

There were 48 probe trials. Thus, not every possible location was accompanied by a probe. For the probe trials, immediately upon either travelling halfway across a large room or entering a new room, people were presented with a probe that appeared in the middle of the screen. People were to respond “yes” if the probe was either the object that was currently being carried or the one that had just been set down. They were to respond “no” to all others. Negative probes were generated by recombining the object and colour name for the two positive objects. For example, if the carried object was a white cube, and the set-down object was a red wedge, a negative probe might be “red cube”. Responses were made by pushing one of two buttons on a computer mouse held in the right hand. Half of the probes in each condition occurred after a spatial shift, and half did not. There were 24 positive probes and 24 negative probes. The experimental procedure typically lasted between 15 and 20 minutes.

Results and discussion

The error rate and response time data are summarized in Table 1. These data were submitted to two-way (shift vs. no-shift) repeated measures analyses of variance (ANOVAs). Because they involve different types of responses, the positive and negative data were analysed separately. For the error rate data, there were significant effects of shift, F(1, 54) = 25.18, MSE = .006, p < .001, and F(1, 54) = 33.86, MSE = .018, p < .001, for positive and negative responses, respectively, with people remembering less after having moved to a new location. Thus, updating an event model following a spatial shift, even for less immersive events, resulted in a memory disruption.

Table 1. Error rate and response time results with standard errors for Experiment 1, which used the small computer displays

Treating responses to the positive items as hits and responses to negative items as false alarms, we calculated A′ signal detection measures. This analysis revealed that memory was better in the no-shift (M = .97, SE = .01) than the shift condition (M = .93, SE = .01), F(1, 54) = 32.13, MSE = .001, p < .001.

For the response time analysis, any errors were excluded. We also trimmed the data by first removing response times faster than 200 ms and slower than 10,000 ms as being impossibly fast and slow. Then, the data were submitted to the van Selst and Jolicoeur (1994) trimming procedure, which is based on the number of observations per cell. This resulted in 6% of the response time data being dropped. There was no effect of event updating for the response time data for the positive trials, F < 1, but a marginally significant effect for negative trials, F(1, 54) = 3.86, MSE = 84, p = .06.

Thus, overall, even with a smaller degree of immersion, there was a location-updating effect. Memory was worse following a location shift. This is consistent with the idea that updating an event model can disrupt memory. In light of the error rate data, the absence of an event updating effect in the response time data is ambiguous. This result cannot be interpreted as reduced forgetting because the pattern of error rates is unchanged. A likely possibility is that the smaller display size reduced the amount of the visual angle (24° vs. 80°) that needs to be actively monitored, thereby speeding response time overall (358 ms faster on average in the current study) and reducing our ability to observe a response time difference.

EXPERIMENT 2

Jump to section

• EXPERIMENT 1
• EXPERIMENT 2
• EXPERIMENT 3
• GENERAL DISCUSSION

The aim of Experiment 2 was to assess whether it would be possible to observe the location-updating effect if one were in a real environment that was maximally immersive. Another way of stating this is to say that a real experience is nonmediated compared to the mediated experiences derived from viewing something on computer screen (Lombard & Ditton, 1997). There is some evidence that the impoverished nature of virtual environments, relative to real environments, can produce deficits in cognition that are tied to that environment (e.g., Richardson, Montello, & Hegarty, 1999; Waller, Loomis, & Haun, 2004). Specifically, a real environment provides a broader range of cues that allow more accurate performance. It may be that it is this paucity of spatial cues that makes the location shift more disruptive in our virtual environments. In contrast, according to an event cognition view, the need to monitor and update an event model should operate in real situations as well.

For the virtual environments of our prior work, people moved through a series of rooms, one after the other, and never returned to a previous room. However, this is not practical in the real world, as it would be difficult to find an environment with over 50 rooms that would allow a person to go from location to location. To adapt the principles of this paradigm to the real world, we did the following. First, we used three larger rooms from our laboratory. There were three location shifts in which a person moved from one room to another. There was a no-shift condition within each room in which a person first did a task at one table and then crossed the room to do the next task. For practical reasons, half of the participants ended their last trial by returning to the original room. To allow for an adequate number of observations, rather than moving a single object each time, there were six objects moved on each trial.

Method

Participants

Sixty people (28 female) were recruited from the University of Notre Dame and were given partial course credit for their participation, with 15 people in each movement condition (see below).

Materials and procedure

A three-room environment was used. The room sizes were: Room 1, 4.4 m × 1.5 m; Room 2, 2.1 m × 3.6 m; and Room 3, 5.2 m × 3.5 m. There were four movement conditions. Half of these began with a no-shift condition and half with a shift condition. Within each of those, one path moved in one direction through the three rooms, and the other moved in the other direction. A plan view of the lab, and the relative positions of the six testing locations, is provided in Figure 1. A set of coloured blocks was used. The block shapes were: cube, sphere, disc, wedge, cross, and cylinder. The colours were: black, white, red, yellow, green, and blue. On each trial, each set of blocks included one block of each shape, and each colour was represented only once.

View larger version(22K) Figure 1 A plan view of the lab and the relative positions of the six testing locations.

At the beginning of each trial, people approached a table that had an inverted black box, covering the objects underneath it. They lifted the box revealing the six coloured blocks and picked up each object and put them into the box. They covered the box with a lid and were directed to the next table on the path where they set the box down. There, the experimenter gave them a laptop computer with the recognition test on it. Prior to the recognition test itself, people were given three-digit maths problems (e.g., 254 + 742 = ?) for two minutes to serve as a distractor and encourage some forgetting. Unlike Experiments 1 and 3 where there was continuous movement through the environment, the blocked presentation of objects may have allowed chunking that would be more difficult in those experiments, hence the motivation to include the distractor task. Note that when a person was seated at a table, their back was turned away from any other tables.

For the recognition test, people saw a series of object names (e.g., “red cube”) and indicated whether the object was in the box that they had just carried. Responses were made by pressing one of two buttons on a computer mouse. The left button was marked with a “Y” for “Yes, this object is in the box”, and the right button was marked with an “N” for “No, this object is not in the box”. On each of the six trials, there were 12 recognition probes. Six were positive (the objects were actually in the box), and 6 were negative. The negative probes were colour and object combinations such that each object shape and colour appeared only once on a given test, and any given colour object combination could appear only once as a negative probe across all of the trials for a given participant.

After the recognition test, the person lifted the next box and began the next trial. This continued until all six trials were complete. The experimental procedure typically lasted between 15 and 20 minutes.

Results and discussion

The error rate and response time data are presented in Table 2. The response time data were trimmed using the same procedure as that in Experiment 1. The data for the positive and negative responses were submitted to two-way (no-shift/shift) repeated measures ANOVAs. For the error rate data, people made more errors following a spatial shift, F(1, 59) = 5.91, MSE = .011, p = .02, and F(1, 59) = 3.69, MSE = .009, p = .06, for the positive and negative responses, respectively. This finding parallels the location-updating effect observed with virtual environments. Note that the larger error rates in this experiment are probably due to the increased levels of proactive interference that people experienced with the increased number of objects on each trial.

Table 2. Error rate and response time results with standard errors for Experiment 2, which used a real environment

Again, treating responses to the positive items as hits and responses to negative items as false alarms, we calculated A′ values. This analysis revealed that memory was better in the no-shift (M = .86, SE = .01) than in the shift condition (M = .82, SE = .02),F(1, 59) = 5.28, MSE = .007, p = .03.

For the response time data, although people were slower following a location shift, this effect did not reach significance, F(1, 59) = 1.34, MSE = 160,662, p = .25, and F < 1, for the positive and negative responses, respectively. Again, this is not much of a concern as the primary measure here is accuracy, and the response times are not inconsistent with the error rate data.

Overall, Experiment 2 demonstrated a location-updating effect in a real-world environment similar to what has been observed in virtual environments. People forgot more following a spatial shift than when they simply moved across a room. This result supports an event cognition view because the need to update one's event model following a change in location brought about a cost in memory performance. Information associated with a prior location became less available even though it continued to be task relevant.

EXPERIMENT 3

Jump to section

• EXPERIMENT 1
• EXPERIMENT 2
• EXPERIMENT 3
• GENERAL DISCUSSION

Another interpretation of the location-updating effect is that this forgetting is due to a difference in environmental context at retrieval compared to encoding. The different rooms are different contexts, and memory may be poorer when the environmental context differs from the original context because there are fewer retrieval cues available (e.g., Smith, Glenberg, & Bjork, 1978; Smith & Vela, 2001). Thus, the location-updating effect would be little more than another demonstration of the encoding specificity phenomenon (Thomson & Tulving, 1970). At the outset this seems unlikely because (a) encoding specificity effects are more reliably observed with recall, rather than recognition as was done here (Smith et al., 1978); (b) encoding specificity generally requires more forgetting to occur for the environmental context to have an effect as a memory cue; and (c) when a carried object was moved from one room to the next, it was then associated with both the original and the new location, rather than just a single context as in typical encoding specificity work. Still, an encoding specificity account does have some plausibility, so we put it to the test.

The aim of Experiment 3 was to assess this alternative explanation. According to accounts of environmental context-specific memory, memory declines when the context changes, making it more difficult to access information in memory. Moreover, and more importantly, when a person returns to the original context, memory should improve as there are now more retrieval cues available to access the information. Thus, in Experiment 3, we added a third condition, the return condition, in which a person, after making a spatial shift, returned to the original location. According to an encoding specificity account, memory should be better in this condition.

One notable aspect of the return condition is that there are two spatial shifts: one when the person moves from the original room to the new room, and then again when a person moves from the new room back to the original one. To parallel this double movement, but not have a return to the original room, we had a double shift condition. In this condition, people moved from the original room to a new room. Then, half-way through the new room, they were told to continue on to yet another room. Thus, people made two spatial shifts, as in the return condition; however, rather than returning to the original room, they were in a new room when they were probed. Thus, these two conditions parallel each other in terms of the distance travelled and the number of spatial shifts, and they only differed in the final location reinstating the learning context or not.

In comparison to an encoding specificity account, for the event horizon model, performance will primarily reflect the number of rooms a person has been in. In the no-shift condition, there is only one room, in the shift and return conditions, there are two rooms involved (thereby producing interference at retrieval), and in the double shift condition there are three rooms involved (increasing the amount of interference experienced). Note that the account is not the number of shifts that disrupt memory (which would predict similar performance in the return and double shift conditions) or that memory disruption occurs after a single shift, but then does not increase (which would predict similar performance in the shift, return, and double shift conditions), but that it is the number of event models (based on rooms in this case) that are involved in retrieval (e.g., Bower & Rinck, 2001; Radvansky, 1999; Radvansky & Zacks, 1991).

Method

Participants

Forty-eight people (28 female) were recruited from the University of Notre Dame and were given partial course credit for their participation.

Materials and procedure

Unlike Experiments 1 and 2, but like our previous work (Radvansky & Copeland, 2006; Radvansky et al., 2010), the virtual environments were presented in a 66″ diagonal Smartboard, with people sitting about 1 metre from the display, to give a fairly high degree of immersion in the virtual environment. For Experiment 3, the virtual space was an 88-room environment. Like Experiment 1, the rooms were one of two sizes, with larger rooms being twice as long as small rooms. For the return and double shift conditions, the intermediate room (where they got the message to go back or to continue on) was always a small room.

Included in each room (except the intermediate rooms, which had no tables) were one or two rectangular tables. Each table was placed along a wall in the room. For the small rooms there was only a single table, whereas for the large rooms there was a table on each side of the room. At one end of the table was the object the participant was to pick up. The other half of the table was empty. This is where the object being carried from the previous room was to be put down. Each room had a different pattern on the walls to emphasize that there was a change in location. Finally, the doorways in the room were never on the same wall.

If the room was one that a person was to return to on a return trial, then there was a third door that would be passed through the second time the room was entered. To reduce the predictability of motion to some degree (i.e., knowing that a room with three doors would be one that would be returned to), there were a number of additional doors throughout the area that never opened.

The procedure was like that of Experiment 1, with a few modifications to accommodate the return and double shift conditions. For the return and double shift conditions, a message was presented to instruct people where to go next. For the return condition, this message told people to turn around and go back, whereas for the double shift condition this message told them to continue on. This message was always presented half-way through the room so that the distance travelled would be similar in the two cases. There were 64 memory probe trials. Thus, not every spatial shift was accompanied by a memory probe. The experimental procedure typically lasted between 15 and 20 min.

Results and discussion

The error rate and response time data are presented in Table 3. The response time data were trimmed using the same procedure as that in Experiments 1 and 2, resulting in 2% of the data being dropped. The data were first submitted to one-way ANOVA with four levels (no-shift/shift/return/double-shift) repeated measures ANOVAs, followed by tests of simple effects. Again, the positive and negative responses were analysed separately. For the error rate data, there was a main effect of condition, F(3, 141) = 6.84, MSE = .011, p < .001, and F(3, 141) = 3.94, MSE = .015, p = .01, for the positive and negative responses, respectively. Simple effects tests revealed that people made marginally significantly fewer errors in the no-shift than the shift condition, F(1, 47) = 2.28, MSE = .007, p = .08, although this was not significant for the negatives, F(1, 47) = 1.51, MSE = .013, p = .23, and significantly fewer than in the return, F(1, 47) = 5.94, MSE = .010, p = .02, and F(1, 47) = 5.10, MSE = .010, p = .03, for positives and negatives, respectively, and double-shift conditions, F(1, 47) = 16.48, MSE = .013, p < .001, and F(1, 47) = 11.19, MSE = .015,p = .002, respectively. Moreover, response errors in the shift condition were similar to those in the return condition, both Fs < 1, but were significantly fewer than in the double-shift condition, F(1, 47) = 6.27, MSE = .015, p = .02, and F(1, 47) = 3.40, MSE = .020, p = .07, for positives and negatives, respectively. Finally, people made fewer errors in the return than the double-shift condition, F(1, 47) = 5.34, MSE = .009, p = .03, although this was not significant in the data for the negatives, F(1, 47) = 1.73, MSE = .011, p = .20. Overall, we replicated the location-updating effect. These effects were weaker in responses to the negative probes; however, this is of only secondary concern as responses to these probes were for items that did not exist, along with the other additional cognitive processes that are known to complicate negative responses. Of primary interest, there was no evidence that returning to the original room improved performance, as suggested by an encoding specificity account. Finally, it should be noted that the large number of errors in the double-shift condition suggests that it is not the number of spatial shifts that disrupt memory, but the number of new areas entered, consistent with the competitive retrieval aspects of the event horizon model.

Table 3. Error rate and response time results with standard errors for Experiment 3, which used a virtual environment

Again, treating responses to the positive items as hits and responses to negative items as false alarms, we calculated A′ values. There was a main effect of condition, F(3, 141) = 10.18, MSE = .005, p < .001. Simple effects revealed that memory was better in the no-shift (M = .94, SE = .01) than in the shift (M = .92, SE = .01), return (M = .90, SE = .01), and double-shift conditions (M = .86,SE = .02), F(1, 59) = 5.22, MSE = .003, p = .03, F(1, 59) = 11.86, MSE = .003, p = .001, and F(1, 59) = 18.24, MSE = .008, p < .001, respectively. Moreover, memory in the shift condition was similar to that in the return condition, F(1, 59) = 1.68, MSE = .003, p = .20, but better than that in the double-shift condition, F(1, 59) = 9.66, MSE = .008, p = .003. Finally, memory was better in the return condition than in the double-shift condition, F(1, 59) = 5.93, MSE = .007, p = .02. Again, the location-updating effect was replicated, and there was no evidence of encoding specificity. Performance was guided more by the number of new rooms entered than the number of spatial shifts.

For the response time data, the main effect of condition was significant, F(3, 141) = 6.49, MSE = 147,724, p < .001, and F(3, 141) = 9.43, MSE = 105,284, p < .001, for positives and negatives, respectively. Simple effects showed that, although the difference between the no-shift and shift conditions was not significant for the positives, F(1, 47) = 1.92, MSE = 121,662, p = .18, it was for the negatives, F(1, 47) = 13.61, MSE = 97,376, p = .001. Moreover, people responded faster in the no-shift condition than in the return, F(1, 47) = 5.47, MSE = 167,254, p = .02, and F(1, 47) = 18.15, MSE = 156,677, p < .001, for positives and negatives, and double-shift conditions, F(1, 47) = 17.86, MSE = 147,662, p < .001, and F(1, 47) = 9.87, MSE = 76,170, p = .003, respectively. Moreover, responses in the shift condition were not significantly different from those in the return condition, F(1, 47) = 2.00, MSE = 111,747, p = .16, and F(1, 47) = 2.37, MSE = 120,937, p = .13, for positives and negatives, respectively, but were faster than those in the double-shift condition, F(1, 47) = 7.09, MSE = 183,518, p = .01, although not for the negatives, F < 1. Finally, responses to the return condition were only marginally significantly faster than those in the double-shift condition, F(1, 47) = 2.89,MSE = 154,498, p = .10, and showed the reverse pattern for the negatives, F(1, 47) = 7.56, MSE = 88,856, p = .008, although it is unclear at this point why this occurred. Overall, the response time data largely paralleled the analyses of the error rates. Of most importance here was that there was no improvement by returning to the original context. Thus, we can confidently reject a context-based account of the location-updating effect.

GENERAL DISCUSSION

Jump to section

• EXPERIMENT 1
• EXPERIMENT 2
• EXPERIMENT 3
• GENERAL DISCUSSION

The three experiments reported here further assessed whether the location-updating effect (Radvansky & Copeland, 2006; Radvansky et al., 2010), in which people show poorer memory for objects after a shift from one room to another. Experiments 1 and 2 assessed whether this effect is influenced by the degree of immersion. In general, this does not play a major role. In prior work, a large display was used that took up most of the visual field. When this was reduced in Experiment 1, a location-updating effect was observed in the accuracy measure. Moreover, virtual environments are not as immersive as real environments. Experiment 2 showed that people in a real environment also showed a location-updating effect. So, a location-updating effect occurred both after increasing and after decreasing the degree of immersion. Thus, the need to update one's event understanding can disrupt memory. Moreover, Experiment 3 explored whether the location-updating effect might be just another manifestation of the encoding principle. However, having people return to an earlier room did not improve memory. Furthermore, this experiment revealed that performance was affected by how many shifts to new rooms there were, not the number of spatial shifts made, consistent with predictions of the event horizon model.

One possible interpretation of the updating effect is that the forgetting of object information during room shifts is due to the disruption of visual–spatial processing in working memory. That is, when one is moving from one room to another, visual–spatial processing that occurs in earlier rooms impedes the visual–spatial processing of the upcoming room(s), thus overloading and adding more and more information to working memory. While this explanation is plausible given the data reported here, results of another study by Radvansky et al. (2010) that used a similar paradigm provides evidence against this processing disruption account. In this study, people were given sets of word pairs to remember (which were unrelated to the object probes) as they navigated the virtual environment and performed the same basic task. A disruption of memory was found following an event shift from one room to another even for these unlinked verbal materials. Thus, the updating effect found in this experiment is unlikely to be due to an exclusively visual–spatial effect.

According to the event horizon model, there are three aspects of performance that are driving the observed pattern of results. First, people are parsing the stream of action into events based on the event boundaries (e.g., Kurby & Zacks, 2008; Swallow et al., 2009; Zacks et al., 2009), which are the shifts from one room to another in the context of the current experiments. This is one reason why information is less available following a spatial shift. Second, information that is being actively processed in the current event is foregrounded and is more available (e.g., Glenberg et al., 1987). Finally, the third aspect of the event horizon model that accounts for the pattern of performance is the idea that there is interference during competitive performance (e.g., Bower & Rinck, 2001; Radvansky, 1999; Radvansky & Zacks, 1991). In this task, when an object is moved from one room to another, it is now represented in two event models, one for the room it was picked up in, and one for the current room. As such, when a person gets a recognition memory probe, both event models are activated. Because recognition is trying to select out a single memory trace, these two event models interfere with one another, thereby increasing the error rates and slowing retrieval time, even though the two models are consistent with the same result. In essence, this is a kind of fan effect (Anderson, 1974; Radvansky, 1998, 1999, 2005, 2009; Radvansky et al., 1993; Radvansky & Zacks, 1991).

So, event parsing, foregrounding, and competitive retrieval, as outlined by the event horizon model, influence performance. These can be separated by assessing performance on the positive probes as a function of whether (a) there was an event shift, (b) the object was either the associated object (the one currently being carried) or the dissociated object (the one just set down), and (c) there were one or two event models involved in retrieval.

The influence of the event shift is to make knowledge about the current event more available and that about the prior event less available as attention moves from one event model to the next (e.g., Morrow, Bower, & Greenspan, 1989; Morrow et al., 1987; Radvansky & Copeland, 2010; Rinck & Bower, 1995). When there is no location shift, there is only one event model involved, and it is for the current location. In comparison, when there is a location shift, then there are two event models involved for the associative information—namely, the room where the object was picked up and the room where the object is being carried.

The influence of foregrounding is primarily on whether the object is currently being carried. Consistent with Glenberg et al.(1987), associated objects are foregrounded in the event model, whereas dissociated objects have been removed from the foreground. This foregrounding results in a higher activation level for those objects. However, when the information moves out of the foreground, its activation level diminishes, making it hard to retrieve such information.

Finally, the influence of competitive retrieval reflects whether there has not been a spatial shift (one event model and, hence, no competition) or there has been a spatial shift (two event models and, hence, retrieval competition). When there is competition, it is expected that there will be an increase in error rates and response time as retrieval becomes more difficult (Bower & Rinck,2001; Radvansky, 1999; Radvansky & Zacks, 1991).

The predicted influence of the combination of various event model processing components and the associate/dissociated object and spatial no-shift/shift manipulation is shown at the top of Table 4. In the prediction row of the table, + indicates a retrieval benefit, and – indicates either no influence or a retrieval cost. The first symbol is for the influence of event parsing and the movement from one event to another, the second is for the influence of foregrounding, and the third is for the influence of retrieval competition. In this way, we consider how each of these components plays out by looking at the four individual conditions that appear across multiple studies.

Table 4. The influence of foreground and competitive retrieval on performance, along with error rate data for five experiments

According to the event horizon model, for the no-shift/associated condition, there has been no shift in location, so the event model where the objects were interacted with is still the one being actively processed. Second, because these are associated objects, they are currently being carried by the person and so are in the foreground of the event model and are more activated. Finally, because there has not been an event shift, there is only a single model involved during retrieval when the memory probe is presented, and so there is no retrieval interference. So, for the no-shift/associated probes, the designation is + /+ / +.

For the shift/associated condition, there is a shift in location, so the event model where the objects were picked up is less available, but because this is an associated condition, the objects have been transported to the new location, which is being actively processed. Second, again, because these are associated objects, they are currently being carried and are in the foreground of the event model. Finally, because there has been an event shift, there are two models involved during retrieval when the memory probe is presented, and so there is retrieval interference. So, for the shift/associated probes, the designation is + /+ / –.

For the no-shift/dissociated condition, there was no shift in location, so the event model where the objects were interacted with is still the active one. Second, because these are dissociated objects, they have been moved out of the event model foreground and so are less available. Finally, because this is a no-shift condition, there is only a single model involved during retrieval. So, for the no-shift/dissociated probes, the designation is + / – / +.

Finally, for the shift/dissociated condition, the event model where the objects were interacted with is less available, and because this is a dissociated condition, the objects were not moved to the new location, and so the event model they are associated with is not being actively processed. Second, these are dissociated objects that are not in the event model foreground and so are less available. Finally, although there has been an event shift, there is still only one model involved during retrieval because these objects were not moved to the new location and so would not be retrieval interference. Thus, for the shift/dissociated probes, the designation is – / – / +.

In Table 4, below the event horizon model predictions are the data from five experiments in which associative/dissociative and no-shift/shift were factorally combined. This includes Experiment 2 from the Radvansky and Copeland (2006) study (Experiment 1 did not have a no-shift condition), both experiments from the Radvansky et al. (2010) study, and Experiments 1 and 3 from the current study (Experiment 2 did not have an associative/dissociative manipulation).

As can be seen, the error rate data, particularly when one averages across the experiments, are consistent with the event horizon model. Specifically, performance is best when the objects are associated, and there has not been a spatial shift. However, when the objects have been dissociated or there has been a spatial shift, then performance is compromised. Thus, this suggests that both aspects of the event updating process are influencing memory performance as one dynamically moves through space. Finally, when there has been an event shift, and the objects were dissociated, there are multiple influences worsening performance. This is also true of all of the individual experiments, except for Experiment 3 of the current study. There is no clear reason why there would be a deviation in the shift/dissociated condition here, and so we tentatively attribute this to random variation.

So, overall, the event horizon model adequately accounts for the availability of objects in an environment that a person is navigating. This availability is influenced by (a) whether there has been an event shift, (b) the foregrounding of currently relevant information, and (c) the presence or absence of retrieval interference.

In sum, walking through doorways serves as an event boundary, thereby initiating the updating of one's event model. This updating process can reduce the availability of information in memory for objects associated with the prior event. Here, we were able to show that this effect extends to different degrees of immersion and is not a result of an encoding specific problem. Finally, an analysis across multiple studies shows that the parsing of the flow of experience into events, foregrounding, and competitive retrieval all combine to influence processing as a function of whether an object is associated or dissociated and whether there has been an event shift. Thus, overall, it is quite clear that memory for recently experienced information is affected by the structure of the surrounding environment.

Acknowledgments

We would like to thank Dan Blakely, Mark Bohay, Abbi Daugherty, Erica Nason, Patrick O'Keefe, Jenny Walls, Megan Cefferillo, and Brittany Gragg for their assistance in collecting the data. We would also like to thank Jeff Smith and Mike Villano for their programming expertise. This research was supported in part by a grant from the Army Research Institute, ARMY-DASW01-02-K-0003 and funding from J. Chris Forsythe of Sandia National Laboratories.

REFERENCES

• 1. Anderson, J. R. 1974. Retrieval of propositional information from long-term memory. Cognitive Psychology, 6: 451–474.[CrossRef], [Web of Science ®], [CSA]
• 2. Bower, G. H. and Rinck, M. 2001. Selecting one among many referents in spatial situation models. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27: 81–98. [CrossRef], [PubMed], [Web of Science ®], [CSA]
• 3. Bystrom, K. E., Barfield, W. and Hendrix, C. 1999. A conceptual model of the sense of presence in virtual environments.Presence: Teleoperators and Virtual Environments, 8: 241–244. [CrossRef], [Web of Science ®]
• 4. Curiel, J. M. and Radvansky, G. A. 2002. Mental maps in memory retrieval and comprehension. Memory, 10: 113–126. [Taylor & Francis Online], [PubMed], [Web of Science ®], [CSA]
• 5. Glenberg, A. M., Meyer, M. and Lindem, K. 1987. Mental models contribute to foregrounding during text comprehension.Journal of Memory and Language, 26: 69–83. [CrossRef], [Web of Science ®], [CSA]
• 6. Kurby, C. A. and Zacks, J. M. 2008. Segmentation in the perception and memory of events. Trends in Cognitive Sciences, 12: 72–79. [CrossRef], [PubMed], [Web of Science ®]
• 7. Lombard, M. and Ditton, T. 1997. At the heart of it all: The concept of presence. Journal of Computer-Mediated Communication, 3 doi:10.1111/j.1083-6101.1997.tb00072.x
• 8. McNamara, T. P., Altarriba, J., Bendele, M., Johnson, S. C. and Clayton, K. N. 1989. Constraints on priming in spatial memory: Naturally learned versus experimentally learned environments. Memory & Cognition, 17: 444–453. [CrossRef],[PubMed], [Web of Science ®]
• 9. Morrow, D. G., Bower, G. H. and Greenspan, S. L. 1989. Updating situation models during narrative comprehension. Journal of Memory and Language, 28: 292–312. [CrossRef], [Web of Science ®], [CSA]
• 10. Morrow, D. G., Greenspan, S. L. and Bower, G. H. 1987. Accessibility and situation models in narrative comprehension.Journal of Memory and Language, 26: 165–187. [CrossRef], [Web of Science ®], [CSA]
• 11. Radvansky, G. A. 1998. The organization of information retrieved from situation models. Psychonomic Bulletin & Review, 5: 283–289. [CrossRef], [Web of Science ®]
• 12. Radvansky, G. A. 1999. The fan effect: A tale of two theories. Journal of Experimental Psychology: General, 128: 198–206.[CrossRef], [PubMed], [Web of Science ®]
• 13. Radvansky, G. A. 2005. Situation models, propositions, and the fan effect. Psychonomic Bulletin & Review, 12: 478–483.[CrossRef], [Web of Science ®]
• 14. Radvansky, G. A. 2009. Spatial directions and situation model organization. Memory & Cognition, 37: 796–806. [CrossRef],[Web of Science ®]
• 15. Radvansky, G. A. and Copeland, D. E. 2006. Walking through doorways causes forgetting. Memory & Cognition, 34: 1150–1156. [CrossRef], [Web of Science ®]
• 16. Radvansky, G. A. and Copeland, D. E. 2010. Reading times and the detection of event shift processing. Journal of Experimental Psychology, Learning, Memory, and Cognition, 36: 210–216. [CrossRef], [PubMed]
• 17. Radvansky, G. A., Copeland, D. E. and Zwaan, R. A. 2003. Aging and functional spatial relations in comprehension and memory. Psychology and Aging, 18: 161–165. [CrossRef], [PubMed], [Web of Science ®], [CSA]
• 18. Radvansky, G. A., Spieler, D. H. and Zacks, R. T. 1993. Mental model organization. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19: 95–114. [CrossRef], [PubMed], [Web of Science ®]
• 19. Radvansky, G. A., Tamplin, A. K. and Krawietz, S. A. 2010. Walking through doorways causes forgetting: Environmental integration. Psychonomic Bulletin & Review, 17: 900–904. [CrossRef]
• 20. Radvansky, G. A. and Zacks, R. T. 1991. Mental models and the fan effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17: 940–953. [CrossRef], [PubMed], [Web of Science ®]
• 21. Richardson, A. E., Montello, D. R. and Hegarty, M. 1999. Spatial knowledge acquisition from maps and from navigation in real and virtual environments. Memory & Cognition, 27: 741–750. [CrossRef], [PubMed], [Web of Science ®], [CSA]
• 22. Rinck, M. and Bower, G. H. 1995. Anaphora resolution and the focus of attention in situation models. Journal of Memory and Language, 34: 110–131. [CrossRef], [Web of Science ®], [CSA]
• 23. Smith, S. M., Glenberg, A. and Bjork, R. A. 1978. Environmental context and human memory. Memory & Cognition, 6: 342–353. [CrossRef], [Web of Science ®], [CSA]
• 24. Smith, S. M. and Vela, E. 2001. Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8: 203–220. [CrossRef], [PubMed], [Web of Science ®], [CSA]
• 25. Swallow, K. M., Zacks, J. M. and Abrams, R. A. 2009. Event boundaries in perception affect memory encoding and updating.Journal of Experimental Psychology: General, 138: 236–257. [CrossRef], [PubMed], [Web of Science ®]
• 26. Thomson, D. M. and Tulving, E. 1970. Associative encoding and retrieval: Weak and strong cues. Journal of Experimental Psychology, 86: 255–262. [CrossRef]
• 27. Vale Software (2003). Valve Hammer (Version 4.0). [computer software]. Bellevue, WA: Vale Software.
• 28. Van Selst, M. and Jolicoeur, P. 1994. A solution to the effect of sample size on outlier estimation. Quarterly Journal of Experimental Psychology, 47A: 631–650. [Taylor & Francis Online]
• 29. Waller, D., Loomis, J. M. and Haun, D. B. M. 2004. Body-based senses enhance knowledge of directions in large-scale environments. Psychonomic Bulletin & Review, 11: 157–163. [CrossRef], [Web of Science ®]
• 30. Zacks, J. M., Speer, N. K. and Reynolds, J. R. 2009. Segmentation in reading and film comprehension. Journal of Experimental Psychology: General, 138: 307–327. [CrossRef], [PubMed], [Web of Science ®]
• 31. Zwaan, R. A. (1996). Processing narrative time shifts. Journal of Experimental Psychology: Learning, Memory and Cognition,22, 1196–1207