Showing posts with label evaluation. Show all posts
Showing posts with label evaluation. Show all posts

Wednesday, August 17, 2016

Reflective Power


Last week I got to do something I rarely get to do--and I find few museums do either.  Over the course of two days, amidst bits and pieces of the ongoing project, the core staff at the Harriet Beecher Stowe Center and I got a chance to reflect on our journey through the reinterpretation of the house, now at the three year mark and nearing completion next year.

I've written often about our work at the Center often, and I know it has resonated with many of you. We've made progress and had great wins, but we also had places where we would have done things differently.  I'll share more specifics about what we learned later, but this is just a post to encourage everyone to squeeze out that time to be reflective about your work, with your colleagues.  How to start? We tried to think about the path of the project which involved diving back into computer files saying, "when was that meeting?" and saying, "Remember when we thought that was a good idea?"

We began the process just with individual note-taking but then decided that a big flip chart map (above, just one piece of what we finally created) was the way to go, helping us think visually about the path, the lessons learned and what we might do differently.

Doing this before the full end of the project meant that it served as a bit of a reward--a chance to appreciate our work together, and to gather our energies for the final push.  If there's one lesson I learned, it's that a thoughtful, creative, interpretive planning process has the potential to transform an organization.  That transformation is not just the story we tell to visitors, but in this case, it has contributed to creating a culture of ongoing learning, of creative problem-solving and one of engaging visitors in a continuous feedback and evaluation loop.

If you want to hear a bit more about the re-interpretation you can listen to Shannon Burke, Cindy Cormier and me on WNPR's "Where We Live." 

A giant bouquet of appreciation to all my colleagues at Stowe!  Below, Shannon, Emily, and Maura embrace our continuous learning over lunch last week, and get a lesson in Pokemon Go from Charlotte, age 9.

Friday, August 31, 2012

I'm California bound--and you can join in!

Next week, I'm very pleased to be headed to San Diego's Balboa Park to be a member of a session on employee engagement at the 2012 Smith Leadership Symposium, Organizational Innovation and the Engagement Equation,  of the Balboa Park Cultural Partnership on Friday, September 6.  A tremendous line-up of speakers, headed by Chip Conley, frequent Ted Talk speaker and hotelier;  Marianna Adams of Audience Focus, Inc; Richard Evans, who directs EmcArts' programs and strategic partnerships;  and Lori Fogarty, director and CEO of the Oakland Museum,  (and me!) will work with participants to  consider questions such as:
  • How do we leverage the creativity we focus on into the way we work?
  • What does engagement really mean?
  • How can our workspaces encourage creative work?
  • What can thoughtful metrics tell us about the synergy we seek to create?   And what do those metrics say about your organizational culture?
  • How are different generations approaching workplace culture and how can a culture accommodate those differences.
There will also be case studies on organizational change,  and I know there will be lots of thoughtful conversation along the way.  But if you can't make it to San Diego--you can still participate online.  In fact,  online participants get a special session, facilitated by The Center for the Future of Museum's Elizabeth Merritt,  to bring ideas  home from the morning sessions.    I'll be blogging after the fact, and trying to squeeze in a few tweets on Friday as well.

I've watched the Balboa Park Cultural Partnership from afar for a while, and it'll be great to get a chance to see their museums and other cultural organizations close up,  and enjoy the work of a thoughtful, passionate collaborative.

To register for the symposium,  online or in person,  or for more information, visit their website here. 
And as always,  if you want to join me for a cup of coffee or a drink, in either San Diego or Los Angeles,  be in touch!

Thursday, June 28, 2012

Experiment in Progress!

What do you think?  About this? or this?  Do you agree?  Do you wonder about?  In developing exhibitions I'm continually challenging museums to get out there to talk to their communities (more to come on a couple projects I'm working on).  And this week in St. John's, Newfoundland,  I saw a great example of the process at the Rooms,  Newfoundland's provincial museum (and art gallery, and archives, all in one place, with a commanding spot atop a hill overlooking the harbor in this small city).

All museum professionals seem to agree that this big overview history exhibits of a place, whatever that place is, are really challenging to develop.   Some people want timelines;  others want lots of text, others want objects that really matter to them, or to see the place where they grew up.  Newfoundland and Labrador (all one province for those of you not from here) is a huge place with a great many stories, so the challenge is a big one.

At the Rooms,  there's a temporary exhibit called Working on History.  It's a bare bones design to share current interpretive thinking on the topic,  put forth already collected visitor feedback,  and along the way, explain a bit about what museums do when we do exhibits and interpret history.   There was something for virtually every kind of learner to respond to in a way they enjoyed.  So here's some of what I saw (and by the way,  thanks The Rooms for letting me take pictures here!)
First, the introductory label sets forth clear expectations in a really brief text.  Exhibit, opening, two key questions, we need your help and feedback. And the informality makes it clear it's not the usual set-in-stone experience. Done!

Six key stories already identified to explore further.  But the text asks for your "words, feelings and ideas,"  not just the open-ended, "what do you think?"   Here's some of the six stories and responses.
Don't like to write?  Here's another alternative:
The middle image is a great reminder to any of us who might be tempted to romanticize childhood stories;  the bottom image depicts St. John's houses, instantly recognizable to residents and visitors.
Don't like to write or draw, but like to have things organized?  How about a timeline?
But those key topics haven't disappeared.  Here's another way of looking at them (words, no images), based on what previous audience work revealed.
But what about those visitors who like the sound of things?  Not an audio installation,  but a chance to share your thoughts, via a paper quiz that you could submit,  testing your knowledge of distinctly Newfoundland words (for instance, one I learned this week, "scrunchions.")
Objects were installed around the outside of the room grouped by the big topic sections.  So visitors got a chance to share feedback there too.  I think the labels do a nice job of modeling possible response, so visitors aren't just facing a blank page.
The museum is experimenting with digital labels so there was a digital label to experiment with--I'll be really interested to see what visitors make of this and how it's eventually used in the exhibit.  It was funny how much less lively this seemed than the rest of the space.
In addition to all the feedback mechanisms,  there were also labels and sections where the museum explained a bit about the process.  A conservation lab was set up and a staff member (not a conservator the day I was there) was on hand to answer questions and a visitor had engaged him in a very lively discussion about fishing issues.  Additional labels talked about storytelling and about using artifacts.  I'd love to see a next steps in this where they talked about and asked visitor feedback on design as well as content.
A colleague and I had an interesting discussion about the limits of visitor feedback in this exhibition that raised more questions than it answered.  What happens to posts that are visitor-generated but fall outside of accepted historical narrative or are more complex politically than a governmental organization is willing to take on?  In crowd-sourcing,  does the crowd produce the most interesting ideas?  How can those outlying but sometimes important ideas be incorporated into the final exhibition?  And how can that final exhibition be lively in the same way as this temporary version?

Much to consider, and a number of ideas I'll be putting to work elsewhere.


Friday, April 20, 2012

The Power of Paper

I'm always packing up paper to take to various workshops. Just today, I put together my paper, markers and scissors for a session at next week's Museums in Conversation conference in Albany.  So I'm used to watching people write on big pieces of brown paper,  stick up Post-It notes and piece together three-dimensional interactive prototypes from construction paper.
But this week, at the Corning Museum of Glass, I was struck by how powerful paper can be when you ask visitors or participants to make use of it.  I was at Corning to hold some visitor conversations around a re-doing of a section of their innovation exhibit,  the section focusing on advances in glass bottle-making.  The staff had known that the section, now more than a decade old,  just didn't quite do it anymore and they've been working with my colleague Christopher Clarke to re-shape the interpretive effort.   He suggested that it might prove illuminating to talk to some visitors about both the current gallery and the proposed re-designs--that's where I came in.
The great team at Corning put together full-size graphic mock-ups and recruited three groups of participants, ranging in age from 7 to over 70.  With each group, we met in the lobby where I distributed simple note-pads and pens to everyone; we then trooped upstairs to the current gallery.  I was interested in seeing what an open-ended approach would do, so I just asked them to note down what they saw,  any issues they saw,  what they liked, and didn't like.  And all of a sudden,  that pad and pencil were really important.  They looked deeper, they wrote down sections of the label text,  they drew pictures--no matter what age they were.  They were intensely serious and focused on their task.  I think the pads and pens were empowering in some way.  We really did want to know what they thought!
All of us then adjourned to a conference room to look at the proposed new graphics.  In the first group,  we had a useful conversation, but the second group was composed of kids from 7-17 and I was a little concerned about participation.  Out came the Post-It notes.  Each participant was asked to put a note by one thing they they found interesting and one they found confusing.  Again, the paper and pen were empowering.  They looked really hard,  they read all the labels, they sort of went back and forth between different elements trying to decide.  And then, because the notes gave me a place to start the conversation,  it was easy for that group (and another group as well) to share their ideas. After all, everyone had participated.  The pads and the Post-Its helped everyone in the group feel that their ideas were equally valued, and provided a safe space to think individually and collectively about the exhibit and design.  Perhaps it's really not the paper, but the idea that we really asked--that we did want to know what our audience thought.

But here's my favorite part of my time in Corning.  When we came in the second day,  there were a couple of additional Post-Its up.  Evidently during the morning,  a staff member named Betty came into the empty room,  saw the Post-It comments,  understood that we were looking for interesting and confusing elements,  shared her own observations and signed her name. Thanks Betty, Corning staff, volunteer participants and Christopher for such a great learning experience.  I'll keep those pens and paper handy!

Sunday, February 6, 2011

The Word from 8th Grade: That was Awesome!

The term focus group often seems scary or off-putting to some groups I work with--but really, it's just a term for listening to what visitors--and potential visitors--have to say.   Last week,  as part of a new IMLS-supported project at the Ontario County Historical Society,  staff members and I spent two days listening to community members talk about the museum and about the concept for the new exhibition, "Greed and Other Human Desires:  The Early History of Western New York."  You can check out the project blog here to learn more.  

Much of the visitor research work in the museum field focuses on larger institutions as those museums have the funds to commit to full-scale evaluation. The work by Reach Advisors, particularly their survey of Connecticut cultural consumers,  begins to bring audience perspectives from all kinds of museums, including small ones, into the picture.  So I was interested to see what our groups had to say both as it related to larger contexts as well as our particular project.

All the conversations were fascinating, but I was particularly struck by the comments of two groups of 8th graders.  These were students,  chosen by their social studies teacher, with a particular interest in history:  many of them were History Day participants and all were headed towards AP history courses. Here are some of their thoughts on what museums do.
 On labels
  • Kids don't really like reading.   [but then some disagreement from several others who did like to read.]
  • They had like the artifact and a small description—not too small, but not like a history lesson.
  • I had to go on a field trip that I liked up until my teacher told me I had to be reading a lot because if I wasn’t reading I wouldn’t learn.  I’ve learned more from museums than having to read right off a wall.
  • Part of a museum is seeing it, otherwise it would be a library.
On computers
  • If you go somewhere with your family, you don’t want to be stuck at one computer screen, you want to be able to pass it around and talk about it.
  • I don’t like them that much.
  • It’s weird that it’s dirty. [this was a conversation about germs]
  • I feel like with the computer—it’s only one answer you can get.
  • At a museum, you clicked on a computer and read about it—it was a cartoon, Ice Age and seemed kind of boring…clicking.
  • There was 100% agreement in both groups that computers were the least interesting part of a museum visit.
On Hands-On Interactives
  • You could get to feel like what they were doing and how they were doing it and what went through their mind.  You get the sense of “whoa, that was hard for them!"
  • Pops out more in my memory when you’re actually holding stuff
  • It’s easy to forget words and pictures;  easy to remember when you’re actually doing stuff;  touching.
They all felt that the use of reproductions for interactives was critical, as it was important to touch and feel.
On the Power of Imagination, Immersion and the Individual Story
  •  Like with the Holocaust, you know that people died, you know these things happened, when you focus on one person, like Anne Frank hiding and stuff, it makes it real.
  • I always find it interesting when I see pictures of a long time ago, to imagine—how things used to be.
  • Being able to go in, see what it was like.
  • The longhouse—that was awesome [at Ganondagan State Historic Site]
  • I like real life examples—if you’re telling about how they dressed,  they had mannequins—the visual was really cool.
  • You can find different answers if you look around.
  • I always like it when an expert can tell you something about it.
Several comments highlighted what's often a shortcoming of local history museums.  The students very much wanted to understand local events in the context of a larger picture,  to understand, as one put it, "More the quieter events during a larger period of time."   That's something many local museums can be better at.

Most boring museum?
For one, it was an art museum.  “The most boringist thing I’ve ever gone to—they’re hanging on the wall.  Art is art.  You stand around with a whole bunch of people, it’s quiet, you can’t even talk to your other family members,  don’t talk, and don’t scuff your feet.”   For another, a sports hall of fame.  "My dad made me go. I just didn’t think anything was interesting. You just stand there and read."
 On the role of parents and museum-going
  • My parents have never really gone anywhere of their own will, actually.
  •  Went to Albany for basketball tournament—and went to state museum. She [my mother]  was really was interested—They [parents] don’t have as much time.  When we want to go, they go, so they just go too.
  • There are some adults that are just naturally interested in history.
  • My dad likes things in the advertisement, that say you can do something.
And how could the museum let you know what's going on?
  • I don’t read the newspaper
  • Facebook, my home page
  • Posters and flyers, because kids go around town;  some kids don’t go on Facebook,  just coming to school you see posters
Facebook generated a fair amount of discussion.  They didn't quite see why a museum would be on Facebook or why they would want to like a museum there.  However,  they were more interested in the idea of seeing historic photos of where they live on Facebook and were most interested when we told them, if their parents granted permission, their group photo would be on the museum's page.  Said one boy, "You should make those blogs and facebook things more known! "

Our conversation also included their thoughts on the topics of greed, survival and ambition--the subject for another post.  But what I hope my readers take away from these great students (aside from the thought that parents have no lives of their own) is that these conversations are easy to do--and that they can easily become a part of a local history museum's work.  

And the how-to:  check out the resources at the Committee on Audience Research as a starting point.  It's critically important to be clear in your own mind about what you want to know and design questions that reflect that.  And it's even more important to LISTEN!  The goal in these sessions is to hear from the group, not to share what the museum's up to or the problems you have.   An evaluation professional can be immensely helpful in the process, but these simple conversations are something a museum of virtually any size can undertake on its own and still learn a great deal of useful information.

All of us on the project team agreed that we all learned some surprising things--and that these students now have a connection to the new exhibit.  It's a terrific two-way street that benefits all.  These simple conversations are just one way to prevent local history museums from becoming those dinosaurs.   Thanks, 8th graders for teaching me something new!

Monday, January 31, 2011

Feeling Bogged Down?

In a post last fall, I wrote about whether local history museums are in danger of becoming dinosaurs.  It became my most read post, by a huge margin, so obviously, there's a big concern about organizations that are bogged down, stuck in the mud.  But there are answers out there. Today,  I want to encourage small and mid-sized US museums--of all types-- to consider taking part in an easy, thoughtful way to get unstuck.  It's the American Association of Museum's Museum Assessment Program.

I've been a MAP reviewer to two separate organizations over the past year or so.  Very different places, very different organizations.  One was a very small history organization in a very small town in the mountainous west;  the other a historic house part of a larger organization in the Mid-West.  Both applied for the MAP program (which does not cost your organization anything) because they knew they could be better.  The MAP process is an easy application, followed by a self-study and a consultant visit.  In reviewing the self-study documents for my two museums, I found that it gave each organization a chance to really think about its work:  about what they do, who they involve, what they collect, and critically, from my perspective, how they connect with their communities.   And I hope my site visit gave them a chance to ask questions, to hear how other museums organize their work and connect with their communities, to brainstorm a bit and consider the "what ifs" that should be a part of all of our work.   The site visit isn't a judgment--it's a chance for you to work with an experienced museum professional and gain her perspective.  After the site visit,  the consultant completes a written report, which can provide important information and guidance as the museum moves forward.

(And by the way, as a reviewer, I've loved the chance to learn about a new museum, meet new colleagues, and see new parts of the country.)

You can apply for one of three MAP assessments:
  • Organizational
    Collections Stewardship
    Community Engagement 
Each one has a slightly different focus--and a different focus on the roots and how your organization grew--and can grow. Check out the website to find out what's right for you.  Still confused?  The MAP staff is ready and willing to speak with you--just contact them at  map@aam-us.org or 202-289-9118.  But don't delay!  The postmark deadline for this year's applications is February 18.   
 Photos from Pie-Town, New Mexico,  FSA/OWI Collection, Library of Congress

Thursday, February 4, 2010

What makes a Great Podcast?


Last week, I sat down with a group of docents at the Hyde Collection, an art museum and historic house with a tremendous collection in Glens Falls, NY.    We were working together, supported by an IMLS grant for new interpretive efforts to make plans for each of them to create a short, two minute or less podcast about a single work of art in the museum.  These are passionate docents, both committed to the museum and with an enthusiastic interest in art.  But podcasts were new to most of them.

We began by listing the things that would make, we thought, a good podcast.  It boiled down to what we called ended up calling The Four Cs:
  • Conversational:  the podcast narrator was easy and fun to listen to--not jargon filled.
  • Content and Context:  you actually learned some information and were able to put the work of art in a larger context.
  • Connection:  the podcast narrator found a way to connect directly with you, the listener/viewer.
  • Concise:  goes without saying, that the podcast needed to be direct and brief.

Then, as a group, we listened to several podcasts and rated them using a rubric based on the Four Cs, rating each area from 1-5.   We had one hands-down winner that got received 5s in every category from almost every listener.   The winner:  the Frick Collection's description of a Rembrandt self-portrait by the director of the Frick, Anne L. Poulet.   Why?  Take a listen and see what you think... Ms. Poulet drew the docents into the painting by her description that helped you look deeper and make connections to their own lives (and grumpy grandpas, perhaps).  The listeners finished feeling that not only did you understand the painting, but also the painter.  And an unexpected result:  several docents said they'd make a special effort to see the painting on their next visit to New York.


We listened to another podcast that stirred some lively discussion.  It was a podcast from the Museum of London, about an alderwood club.    According to their website, this and other podcasts were specifically designed for visitors with visual impairments, but are suitable for all visitors.   These podcasts were developed as part of the museum's social inclusion program:
Podcasts from the Past worked one day a week for 8 weeks with a small group of adults who are currently long-term unemployed, to create a series of podcasts for visually impaired visitors. The participants are a range of ages, with a rich variety of backgrounds and life stories, but came together to work as a team to realize their abilities and gain news skills and experience they can use in their futures.
The Hyde docents liked the informal tone of the speaker, and some loved the sound effects and others really disliked them--but all agreed, that if you were a young visitor, the sound effects (listen for that thwack at the end) would be memorable!

I hope this meeting had several take-aways for the docents, but I know it did for me.  I gained a handy tool to think about audio tours (and perhaps those Four Cs would serve any label-writer well).  It also reinforced for me a process of working with docents, encouraging them to become active learners themselves in critiquing and understanding the many ways museums connect with visitors.

Photos top to bottom:
Listening to our audio tour, from nicolelikestarts photostream on Flickr
Self Portrait, Rembrandt van Rijn,  from the Frick Collection
Museum of London podcast recording session

Saturday, April 25, 2009

Inside Chernobyl: Life Goes On



The title of this post is the title of the exhibition of photographs by Michael Forster Rothbart, supplemented by some important additions by Chernobyl staff member Alexander Kupny, that opened yesterday in Schvencko Park in Kyiv. For my Ukrainian readers--it will be up in the park for the next two weeks, so please stop by to see it.

For me, some take-away lessons and reminders from this project:



People Matter
Many Ukrainian museums present history in the abstract. This exhibition takes a different, more direct and personal approach. Michael and his colleagues focused on five families who work at Chernobyl today. So the exhibit tells the story of Slavutich, the community where they live, and the Chernobyl plant, through sharing a bit of their lives. Life does go on at Chernobyl, to the great surprise of many here in Ukraine and throughout the world. Today, more than 3800 people still work at the plant. It was wonderful that several of the families were able to make it to Kiev for the opening and to watch them take a look at themselves on the panels in the park. Very pleased, I think.



People Matter Part 2
Any exhibit project is never done by a single person. With financial support from the US Embassy and the Chernobyl Shelter project--and of course the Fulbright Program that made it possible for both Mike and I to be here--Michael worked with a loosely formed collective that ranged from Oleh, a graphic designer in Kharkhiv; to Vasily, whose attention to detail --and those many small nuances--made the production of banners and stands a success; to volunteers Irina, who undertook complicated and sometimes endless seeming negotiations for the site and a million other issues and Natasha, who coordinated the publicity and put together a great group of other young volunteers to help out; Anna, who worked on complex translations, and many others, including staff from the US Embassy, who provided critical assistance. And of course, a fruitful day of brainstorming by my students at Kyiv-Mohyla helped jump start our thinking in new directions. In any project, I always learn lessons about working with people--this was no exception. (My apologies for first names only--I need to check last name spelling, the bane of my existence here, of all the team.)



People Matter Part 3
This project was a great example of why I love doing exhibit projects. It's not for me, or for the museum--but for the audience. Even before we put all the panels up, passersby were stopping by to read--really read--the text and look at the images. People stopped and talked to us about the exhibit and talked to each other. Watching people engage with the stories of Slavutich residents (Slavutichians?) was wonderful--exactly what I hoped for, when, back on a dark January day, I said, "Michael, wouldn't it be fun to have it outdoors?" It is fun.



People Matter Part 4
It's rare for museums here to do any sort of exhibit evaluation. We really wanted to know what people think about this project, so volunteers are conducting survey interviews over the next two weeks. I'll post more about the results when I get some of the feedback and I'll have some of the information for my Idea Lounge session next week at AAM (Friday, 8:00 AM).

And, People Matter Part 5
It's always important to take time to be proud of finishing an exhibit--it's the time when all those bumps along the way begin to recede. We were very pleased that the US Ambassador to Ukraine, William Taylor, the mayor of Slavutich, and other dignitaries joined us to celebrate the exhibit's opening.



Top to Bottom:
Viewing the exhibit panel about Slavutich
Michael thanking Irina Leonenko
Crowd views the exhibit
Surveyors at work
Michael and dignitaries

Sunday, December 7, 2008

The Story of La Guerra Civil or Why I Work in Museums



Nina Simon wrote a fascinating thoughtful post on her Museum 2.0 blog on why she works in museums and encouraged other museum bloggers to do the same. So I'd been thinking about what I would say when an experience this past week brought it into clearer focus for me.

At the beginning--like a number of museum professionals I know--I came into the field as a teenager. I didn't work wearing a costume, but I began volunteering at my local historical society when I was fourteen. I started with my mother's encouragement, mostly as a way to get me out of the house (my older sister was a candy-striper at the hospital which was not so interesting to me). I was lucky enough that summer to work for a young graduate of a museum studies program , who was very encouraging and gave me a varied selection of tasks to do. That year, or one of the ones right after, I did an inventory of the costume collection--and so at first glance, it was the cool stuff that attracted me to the field. I still remember a magenta evening dress from that summer.



But now, as I think about it more, I think the real answer might lie somewhere else--and the illustration for this isn't me in the attic inventorying costumes, but rather me, creaking my bedroom door open so I could read, late after bedtime, by the light of the hallway that streamed in. I loved reading, and I particularly loved a whole range of old-fashioned stories--Anne of Green Gables, the Railway Children, The Lion, the Witch and the Wardrobe. Those stories and the details in them--the apple tree outside Anne's window or the Turkish delight found through the wardrobe door. These compelling stories, filled with detail and texture, fill my memories. As my career in museums has progressed, it's that love of stories, that opportunity to understand other lives, that has formed the main frame for my work. I found I wanted to tell stories, to hear stories, and to connect stories to other lives, past and present.

What reinforced it this week? For a project at the Montgomery County Historical Society in Rockville, MD, evaluator Catherine Harris conducted a series of focus groups for us about different topics. Montgomery County is tremendously diverse (at some high schools, 94 different languages spoken). This project will be done tri-lingually, in English, Spanish and Chinese, the three most prevalent languages in the county. The Spanish-speaking group, all mothers meeting at a high school, provided me with a memorable connection. We asked participants about their interest in several different topics in local history. One topic was the Civil War, with a question framed around the idea of battles fought in your own backyard. I have to admit, sometimes I think of the Civil War as the province of re-enactors, or academic historians.

But to these women, most from Central America, the idea of the Civil War here was a fascinating, important one. Why? To paraphrase one participant, it was because they knew civil war at home, where brothers and fathers were taken or killed, and it was amazing to think that, not much more than a hundred years later, that a place where the American Civil War was fought could now be this--the land of highways, rolling fields, housing developments and shopping malls--and of freedom and safety.

After this morning discussion, I wanted not only to find better and more meaningful ways to tell the story of the Civil War, but also to learn the stories of these particular women and find ways to share them with their community. Relevance comes in surprising forms--but only if you take the time to listen to your audience. Those human stories connect us in many more ways than we might perhaps imagine.



Top and Bottom: Participants in the Spanish language focus group, Gaithersburg, MD
Center: Boys read story books in the shade, Caldwell, Idaho, July, 1941. Photograph by Russell Lee, FSA/OWI Collection, Library of Congress

Saturday, September 20, 2008

It's Never Too Late to Evaluate!



I facilitated a session at the AASLH conference called, Six Museums, Six Years, Six Exhibits, about a collaborative project of six Finger Lakes organizations to develop six exhibitions about summer in the Finger Lakes (you can learn more about the exhibits here). Museum educator Mari Shopsis gave a terrific overview of the evaluation she developed for the project. I wanted to recap her talk here as a great example of how small museums can undertake an evaluation process.

We came to evaluation fairly late in the process, after some extensive planning and then a hiatus while more funds were generated. Mari reminded me that her first questions about the project were variations of, "are you serious? Are you prepared to listen to what people have to say? Is there still any room to make changes based on the evaluation? Yes, I assured her, with some trepidation as I thought about the schedule. After some discussion, our goals were to:
  • Determine whether visitors understand exhibit titles & themes in the same way that exhibit team intended them
  • Provide a picture of potential visitor concerns, interests, and associations with exhibit themes.
  • Provide visitor feedback for exhibit team, allowing team to refine concepts and presentation strategies before finalization and fabrication
We ended up with three types of evaluations that fell somewhere between front-end and formative:
  • One-on-one survey & assessment of visitor interest, administered in three cultural/museum sites across the Finger Lakes
  • Prototype interactive components with family and youth groups
  • Use likely visitor focus group to assess success of “tweaked” titles and concepts
All of these methods were useful and all revealed different types of information. The one-on-one survey--which was done at locations different than the six museums--really gave us information about what people thought the exhibits would be about, based on our descriptions, and how interested they would be. We used that information to re-shape several titles and interestingly, found that the exhibits of greatest interest to the project team were the least interesting to the public (a cautionary tale for all of us). In practical terms, it meant we had to work a bit harder to make those particular topics--issues of public access and the use of Native American imagery to promote the region--more interesting and accessible.

After that initial work Mari then met with a small group to discuss the tweaked titles and concepts. From that discussion we framed all six titles, and all were a From ...To.... construction, such as From Steamboat Landing to State Park, to give a clear sense that these were historical exhibits with a contemporary connection. And yes, they all also had colons, which we had hoped to omit.

I've written about the interactive prototyping in an earlier post. The format Mari developed was incredibly helpful in refining both language and activities. As a result, our final simple interactives connected more strongly with family audiences.

Mari ended her presentation with three conclusions that provide great reasons for evaluation in projects. The process doesn't have to be elaborate and complex, but I continue to be amazed at how few museums choose to interact with their visitors and potential visitors in this way. It's not that hard, but does require a commitment to the process and a willingness to listen--and act upon--feedback.

So remember:
  • It’s never too late to evaluate!
  • Evaluation can help to identify themes that resonate for visitors & concepts or wording that confuse or mislead them
  • In large collaborative projects, an outside insight into the visitor’s perspective can provide objective information to resolve conflicts or help make difficult decisions
Above: Labor Day sack race at a cottage community on Seneca Lake, 2006, Photo by Drew Harty