Team

Jason Wang, Nina Gao
Xuan Liu, Irene Yang

Role

User Research
MR/AR Experience Design
Interaction/Interface Design
Habitat and Character Modeling
Animation Prototype
Interactive Prototype
User Testing

Tools

Figma
After Effects
Solidworks (3D CAD)
Keyshot (Rendering)
Adobe Aero (Prototyping)

Timeline

Oct - December 2022 (6 Weeks)

Design objective

The Carnegie Museum of Natural History (CMNH)'s mission is to deepen the understanding of the natural world and embrace responsibility for our collective future; however, our interviews have shown how the exhibit seemed to diverge from the museum's values.
‍
How can we build an immersive and connective experience in CMNH to raise awareness among the community about conservation through technology?‍

Overview

To address this problem statement, our project aims to design a Mixed Reality-based interactive exhibit for the Carnegie Museum of Natural History (CMNH) that will engage visitors and educate them about animal conservation efforts. To enhance the learning experience for visitors, our project aims to incorporate immersive interaction and learning-by-doing elements. By actively participating in the exhibit rather than just reading about it, visitors can better retain the information being presented and understand the importance of conservation within the community. The goal of our project is to create a meaningful and memorable museum experience that encourages visitors to learn more about the natural world and take action to protect it.

OutCome: concept video

OutCome: Key interactions

01 Onboarding and Personalized Navigation

Lightweight augmented visuals in the menu guide visitors through basic gestural tutorials and a designed route or a personalized one.

02 Education: Learning with Interaction

3D interactions can be manipulated to make information available through discovery and 3D Habitat projections are immersive and explorable for visitors to experience the animals’ actions.

03 Form responsibility and connection

Help unlock a better habitat for endangered animals and discover the possibilities when the animals come out of the diorama to interact with the visitor.

04 Collecting and Sharing

Help unlock a better habitat for endangered animals and discover the possibilities when the animals come out of the diorama to interact with the visitor.

mission statement

Deepening understanding of the natural world, embrace responsibility for our collective future

The museum’s missions are raising awareness, education, and community. And all of these feed into their mission statement. We took these values into account in our primary research.

research

CMNH's mission clashes with what the visitors interpret from the exhibit

Our interviews seemed to diverge from the museum's values. Visitors found that they did not retain much educational information due to the heavy text cards. They also felt that the theme of animal conservation did not come through the dioramas. Some interesting points we found were that people said that they felt distant from the animals due to the distance of the glass and that they felt more connected and related to the animals that were familiar and local to them.

“Animal conservation? Where? I cannot tell what is related to animal conservation.”

“I feel distant from the animal due to the glass, I am not immersed.”

“Museum information is outdated and text-heavy.”

Brainstorm and Synthesis

We grouped our observations and interviews by category and synthesized them according to the framework on the right. Turning what we heard into what we learned, which later informs our design implications.

objective

NVISION amplifies the CMNH's mission

We aim to amplify the museum’s missions through: interactive education, emotional connection, and spreading the word of animal conservation through social media.

design directions

design objectives

Form emotional connections through hyper-localized curation and immersive experience.

An accessible and interactive information system for visitors to learn from the museum.

Extend the connection to endangered animals worldwide and motivate caring towards animals.

Promote our exhibition and the importance of animal conservation to a wider community.

Journey Map

Visualizing what a visitor goes through in NVISION

NVISION has 5 different stages of experience design: before the visit, entering the museum, during the visit, exiting, and after the visit.

system diagram

Mixed Reality Experience via Hololens

NVISION requires a HoloLens for the interactions at the museum mixed with AR reality with smartphones for our at-home experience because we want the visitors to have a fully immersive experience with the animals. This immersion aspect is important to us because we want to use a connection to build our visitors' inclination towards animal conservation.

design: AR/MR

Learning about the animals using AR/MR

Visitors will receive postcards depending on which animal for animals they helped during the visit, and they will be able to bring these animals to life again. They will download the museum app with their smartphones and scan the animal illustration on the postcard to have the animal appear in 3d on the cards.

learning the geography: where they lives

learning the food chain: what is the diet (unmute for sound)

learning about conservation: habitat (unmute for sound)

learning about responsibility: how we can do better

design: seeing the gorilla

Building a gamified experience as a reward

We first identified the goal of the exhibit: to build a connection between visitors and the animals and encourage visitors to take action for animal conservation. To achieve this goal, we wanted to create an interactive and immersive experience that would be both fun and educational.

To design the experience, we used a combination of technology and design elements. First, we used head tracking technology on the Hololens to allow the gorilla to react or imitate the movements of the visitor.

We also designed the chest-beating experience to be a hidden gem, with no visual indicator to encourage visitors to explore and discover it on their own. By adding this element of surprise, we aimed to create a sense of excitement and curiosity for visitors. We hoped that by creating this sense of connection between the visitor and the gorilla, visitors would feel more invested in the conservation efforts and be more motivated to take action. Overall, our goal was to create a fun and interactive experience that would engage visitors and inspire them to care about animal conservation.

Reward + hidden gem: chest beating

design: tangibles

Continued experience on the Post Cards

Design: Mobile APp

Continued experience on mobile app

We designed a mobile app that allowed users to initiate an augmented reality experience by scanning the postcards with their phones. The app provided a continued experience for our project, enabling users to engage with it in a more immersive and interactive way. Consequently, they will be able to share the recording to promote the museum.

Home

AR Scan

Record AR

Share recording

prototype: character

Animation in Unity and Maya

We used the 3D model of a gorilla in Unity and attached it to a character controller, which allows us to specify how the avatar should move and behave. We programmed the controller to make the gorilla walk, idle, and chest beat using animation clips and state machines.

prototype: habitat

3D Rendering in Keyshot

We have created our own habitats for the featured animals in our video.

prototype: animal + habitat

Studying Animal in the habitat

We brought 3D models of animals into the various 3D habitats we built into virtual reality. This allowed me to study the space and dimensions of the habitat in a more immersive and interactive way, giving me a better understanding of how the animals would fit and behave within the virtual environment. I used the VR headset to adjust the ratios and positioning of the animals to create the most optimal experience for users. By using VR technology, I was able to create a more realistic and engaging virtual habitat that allowed users to learn about and interact with the animals in a way that was not possible before.

prototype: mr/AR in Adobe Aero

Immerse the audience in the MR diorama

Using Adobe Aero, we created an AR habitat for a museum experience that allowed visitors to learn about different ecosystems and animal species in an interactive and immersive way.

prototype: AR on the Tangibles

Playing with the Post Card AR Models

Using Adobe Aero, we created AR models that could be activated when users scanned the special postcards with their phones. By combining the physical postcard with the digital AR experience, we were able to create a unique and memorable experience that left a lasting impression on visitors.

prototype: AR user interface

Studying optimal text sizes in Augmented Reality

During a Zoom session with Designer John Mars, we showed snippets of our current concept video as well as the onboarding and UIs for AR. From what we observed and learned, we enlarged the text size of our menu, cards for optimal legibility. And changed the many information cards into Sound or VUI to avoid excessive reading.

Studying Viewing Distance: Minimum Text size

AR Interfaces with text

Sound Card

what next?

Reflecting on this project

Overall, I believe that this project was a success. We were able to design an interactive exhibit for the Carnegie Museum of Natural History that engaged visitors and educated them about conservation efforts. By incorporating immersive interaction and learning-by-doing elements, we believe NVISION could create a meaningful and memorable museum experience that encouraged visitors to learn more about the natural world and take action to protect it.

If we had more time and expertise, we would have liked to create a fully interactive prototype using Unity and test it using the Hololens. This would have allowed us to have a better understanding of the overall experience and make any necessary adjustments before implementing the exhibit at the museum. In the future, I hope to continue working on projects that have a positive impact on society and the environment. Using advanced technologies like Mixed Reality, I believe that we can create even more immersive and engaging experiences that will inspire visitors to take action and make a difference.

Enhancing emotional connections and maximizing learning about animal conservation through Extended Realities at the Carnegie Museum of Natural History.