Is it working
•
1 j'aime•880 vues
Panel at AAM2013 regarding evaluating the effectiveness of mobile apps in museums. This slide deck represents the opening presentation, introducing the speakers, my company's research into real space social engagement, and the criteria for evaluating success that our panel identified. Other presentations in this session: http://www.slideshare.net/LoicT/120520-loic-aam-apps-effective-ss-22423632 http://www.slideshare.net/nancyproctor/evaluating-mobile-success-for-aam2013
Recommandé
Contenu connexe
Tendances
Tendances (12)
Similaire à Is it working
Similaire à Is it working (20)
Plus de nightkitcheninteractive
Plus de nightkitcheninteractive (10)
Dernier
Dernier (20)
Is it working
- 1. Is it Working? Evaluating the Effectiveness of Mobile Apps 5/21/13 ?
- 2. How do we define success?
- 3. Introductions • Matthew Fisher, Night Kitchen Interactive (chair) • Loic Tallon, Pocket-Proof • Nancy Proctor, Smithsonian Institution • Matthew Petrie, Fusion Analytics
- 4. Rousing the Mobile Herd Apps that Encourage Real Space Engagement www.whatscookin.com/mobileherd @mefisher
- 5. American Folk Art Presents: "Infinite Variety" American Museum of Natural History: Cosmic Discoveries American Museum of Natural History: Dinosaurs American Museum of Natural History: Explorer Aquarium of the Pacific Asian Civilizations Museum: Terracotta Warriors Balboa Park: Spotlight Mobile Bean Life Science Museum at Brigham Young University: Identify Me Bean Life Science Museum at Brigham Young University: Dichot Key British Museum: Book of the Dead Brooklyn Museum of Art Canadian Museum of Civilization Canadian War Museum Centre Pompidou China Heart City of Calgary Downtown Public Art Circuit Coastal Maine Botanical Gardens Crystal Bridges Museum of American Art: CBMuseum Denver Art Museum: DAM_SCOUT Denver Museum of Nature & Science Graphic Design Museum Guggenheim Bilbao Guggenheim: Cattelan Hammer Museum High Museum: Art Clix Jewish Museum: Radical Camera LACMA Memphis Brooks Museum of Art Milwaukee Art Museum Minneapolis Institute of Arts Minneapolis Institute of Arts: iAfrica Missouri History Museum: Historic St. Louis MoMA Musee des Beaux Arts de Lyons: MBA Lite Musee du Louvre Museum Hunt Museum of Modern Art Vienna: Mumok Museum of London: Streetmuseum Museum Victoria: Field Guide to Fuana Museum of Science: Then and Now National Gallery NICH: eMuseum NLS: Explore the John Murray Archive NMS: Concorde App National Naval Aviation Museum National September 11 Memorial & Museum Nezu Museum New York City Museum Guide (Star and Light) Norton Simon Museum Pace Gallery: 50 Years at Pace Powerhouse Museum: Lovelace Powerhouse Museum: PHM Walks Prado Museum: Audio Guide Queensland Art Gallery of Modern Art Rijksmuseum (Amsterdam) Royal Ontario Museum Guide SI NASM: PixPop SI: Access American Stories SI: Meanderthal SI: Stories from Main Street SI Mobile SI NMAI: American Indian Magazine Solomon R. Guggenheim Museum Storm King Art Center Strawberry Banke Museum Suntory Museum of Art Tate Modern: How It Is Tate Modern: Magic Tate Ball Tate Modern: Magritte Your World Tate Modern: Muybridgizer Tate Modern: Race Against Time Teylers Museum: Gadgets and Games The Albuquerque Museum of Art and History: ABQ Museum Painter The Field Museum: Specimania The Metropolitan Museum of Art: Met Guitars The Metropolitan Museum of Art: Murder at the Met The State Hermitage Museum Toledo Museum of Art University of Melbourne: Formative Histories Walk 75+ museum apps in iTunes store @mefisher
- 7. Things we do in real space with other people
- 8. Social engagement supports more effective museum experiences conversation is a primary mechanism of knowledge construction and meaning-making. Lienhardt & Crowley, 1998 groups observe each other to learn, to understand exhibit interactions and model behavior. Falk & Storksdieck, 2005 @mefisher
- 9. Talking about the exhibit Games in the space Facilitated discussions Touch-tables & multi- person interactives Photographing/observing others
- 10. Meeting some goals… • Engaging visitors with objects • Providing context and insights • Social sharing • Online engagement @mefisher
- 11. …while not meeting others Rousing the Mobile Herd Apps that Encourage Real Space Engagement www.whatscookin.com/mobileherd @mefisher
- 12. How do we define success?
- 13. How do we define success? • Usage analytics • Visitor Participation • Critical Assessment • Serving the Mission • Bottom Line
- 14. Usage analytics • Downloads • Visitor tracking • Pre-, during and Post-visit usage • Mobile visitors compared to online and onsite visitors (quantity, demographics)
- 15. Visitor Participation • Commenting • Social Sharing • Contributed content & crowdsourcing • Engagement pyramid: – Curating – Producing – Commenting – Sharing – Watching
- 16. Critical Assessment • Press reviews • Social media reviews • Reviews/ratings in app stores • Visitor surveys • Compared to other apps
- 17. Serving the Mission • Innovation & experimentation (functionality unique to the platform) • Integration with museum offerings (interpretative and educational) • Integration with museum systems (collections, website, social media) • New audience reach (reaching target or underserved demographics) • Improving audience diversity (cultural and socio-economic, multi- lingual, accessibility) • Onsite and beyond-the-walls
- 18. Bottom Line • Cost per user (compared to other offerings) • Sustainability (ease/cost of maintenance) • Translation to ticket sales or contributions
Notes de l'éditeur
- Thank you for joining us this morning. This session poses the question we all want to answer about our mobile apps. Is it working? But in order to answer this question we must first answer a much more complex one:
- How do we define success? Before going into that, I’d like to introduce our session panelists.
- I am Matthew Fisher, president of Night Kitchen Interactive. Our panelists are LoicTallon, of Pocket Proof, Nancy Proctor, Head of Mobile Strategy & Initiatives at the Smithsonian, and Matthew Petrie, of Fusion Research and Analytics. Each of the three of our panelists brings a unique and compelling perspective to the questions of how and perhaps more importantly what do we measure in determining the success of museum mobile apps? Before we hear from our panelists, allow me to briefly provide my perspective on this question and why it led me to not only organize this session,
- but to co-author a paper for the museums and the web conference last month entitled Rousing the Mobile Herd: Apps that Encourage Real Space Engagement.
- I became interested in this topic last summer when, together with my team at Night Kitchen, we downloaded over 75 of the top museum apps in the app store in an effort to familiarize ourselves with the current state of the industry. My primary interest was in understanding how these apps fulfill what I see as a key strength that museums hold in the continuum of informal learning environments.
- Museums and their peers are unique in that continuum, in that they inhabit the rare intersection of real space and social space. They provide direct access to material culture and natural phenomena on the one hand AND they support a wide range of social behaviors on the other. In short, they are what I call “real social”.
- To tease out this concept of “real social”, we mapped social behaviors into four quadrants with anti-social on the left, social on the right, the virtual below and the real above. Those activities in the lower left quadrant, that are both anti-social and virtual, take the least advantage of the museum space. Imagine, for example, a tween sitting on a bench playing angry birds. Those activities in the upper right quadrant, that are both social and real, take the greatest advantage of the museum space. Imagine a family on a walk through the nature center, sharing thoughts and reflections as all five senses come alive with the experience.
- But don’t take my word for it. As you probably know, there is a wellspring of research identifying social engagement as a key ingredient in effective museum experiences, as our paper explores in greater depth. I’ll simply reference two key points, one being that: “conversation is a primary mechanism of knowledge construction and distributed meaning-making”, and the other, thatgroups observe each other to learn, to understand exhibit interactions and model behavior.
- To promote“real social” engagement, we considered the types of behaviors to be encouraged in the museum space, such as talking about the exhibit, playing a game in the space, engaging in facilitated discussions, participating in multi-person interactives, and photographing or observing others. Yet when we analyzed the apps that were available last summer, we found that the vast majority of them provided features that did not promote real social engagement. Instead, by and large the apps provided features that were anti-social and virtual, such as audio and media tours, single-person games and research, and features that fell into the social and virtual quadrant, such as social sharing and commenting.
- In our paper we were not questioning the benefits that many of these apps provide. We would say that the majority of them effectively meet the goals of engaging visitors in the space, providing visitors with context and insights, and allowing visitors to share and engage with others online. But on a cautionary note, we would say that the more that museums define success in terms of app usage and social media metrics, the more we will design apps that encourage virtual, and often anti-social, behaviors, rather than “real social” behaviors.
- In our paper we simply suggest that IF you agree that “real social” engagement is a key strength for museums, and if this sort of visitor engagement is central to your museum’s mission, then its important to design mobile apps with real social goals in mind, and measure the effectiveness of your apps in meeting those goals. There are several examples in our paper and available on slideshare that I will not go into here.
- So, back to the question at hand: How do we define success?
- There are many ways to do this, so many, in fact, that it is difficult to know where to begin. Our panel gathered a range of metrics and I have attempted to place them into categories, including usage analytics, visitor participation, critical assessment, museum goals, and the bottom line.
- There are usage analytics, such as downloads, visitor tracking, and pre, during and post visit usage comparisons and comparisons of online and onsite visitors.
- You can look at visitor participation, including in-app commenting and social sharing, contributed content when solicited, or even across the entire engagement pyramid.
- There are also critical assessments, including press and social media reviews, the quality and quantity of reviews and ratings in the app stores, visitor surveys, and tracking against other apps.
- There are also many museum goals to track against, including the desire to innovate and provide unique functionality, integrate with museum offerings and systems, reach new audiences and improve audience diversity, and meeting both onsite and beyond the walls requirements.
- And finally there are some bottom line considerations, such as what is the cost per user, is it sustainable, and does mobile usage translate to ticket sales and contributions?