Using Mobile Location‐Based Augmented Reality to Support Outdoor Learning in Undergraduate Ecology and Environmental Science Courses
Amy M. Kamarainen, Joseph Reilly, Shari Metcalf, Tina A. Grotzer, Chris Dede · Bulletin of the Ecological Society of America · 2018
Augmented reality (AR) applications use a technological device to visually display digital information so that it appears to be overlaid, embedded in, or activated by the physical environment. Augmented reality is an emerging technology that falls on a spectrum between real and virtual (Milgram and Kishino 1994); current descriptors of points along this spectrum include mixed reality, AR, and virtual reality (Klopfer 2008, Liu et al. 2017). While AR is currently most visible in entertainment and gaming industries (e.g., Pokemon Go), there is growing theoretical and empirical evidence that AR supports learning and engagement (Price and Rogers 2004, Dede 2009, Radu 2014; Reilly and Dede, in press), and it is important that environmental educators for all ages consider the opportunities and challenges these technologies present (McCauley 2017). Because AR is an emerging medium, there are a number of formats that fall under the AR "umbrella," and we describe the relevant distinctions below. There are two primary formats for AR—location-based and vision-based AR—and each offers different opportunities to support learning (Dunleavy 2014, Dunleavy and Dede 2014). There are also two primary modes for delivering AR experiences—through mobile devices (like smartphones and tablets) or through head-mounted displays (like the Microsoft HoloLens; Radu 2014). Vision-based AR allows a designer to link digital information and media with a physical "trigger," which might be an object, image, or Quick Response (QR) code (like the black-and-white square shown in Fig. 1a). The camera(s) on a smartphone or tablet, or on a head-mounted display (like Microsoft HoloLens), are used to recognize the pattern of the trigger and activate the associated information and media, which is then displayed to the user. This works in a similar way to a barcode that is scanned at a grocery store in order to reveal the price of an item. In contrast, location-based AR involves learners using GPS-enabled smartphones or tablets to activate media at particular locations in an outdoor space (Fig. 1b). A designer uses a map-based online interface to embed digital information and media at locations of interest, and the embedded information or media is activated when the user reaches that location. After being activated by location or by a vision-based trigger, the AR application superimposes digital media, data, audio, video, art, and/or narratives on the real world, making this information appear to be embedded or overlaid on the real environment. Prior work on the use of AR in undergraduate teaching and learning contexts has focused largely on vision-based applications of AR. Studies in this area show that vision-based AR can support student understanding of concepts that require abstraction or interpretation of complex spatial relationships—concepts for which visualization is a useful tool (Radu 2014). For example, work by Lin et al. (2013) demonstrates that undergraduate students learned more about elastic collisions using an AR application for physics compared to a 2D simulation, while Shelton and Hedley (2002) report on improvements among undergraduate geography students in their factual and conceptual understandings of complex spatial concepts associated with Earth–Sun relationships following use of an AR display. Also, when applying vision-based AR to physics and astronomy laboratories, the AR treatment had positive effects on students' attitudes, skills, and conceptual understanding-related specific concepts (Yen et al. 2013, Akçayır et al. 2016). While these studies represent a valuable "proof of concept," more work needs to be done to characterize how AR interfaces may support learning in undergraduate classrooms, and to identify the limits in the utility of AR. One limitation of the prior work is that much of this has been done in indoor learning environments with vision-based AR, and the potential for location-based AR to support learning among young adults in outdoor contexts has been under-studied. Studies in mixed-reality contexts suggest that overlaying digital information with real physical objects can help support transfer by bridging abstract and concrete forms of understanding (Quarles, Lampotang, Fischler, Fishwick, and Lok 2008). Augmented reality offers similar opportunities to bridge between the abstract and concrete (Rogers 2004), and for learners to build deeper connections with the material by learning about otherwise hidden physical, historical, and cultural aspects of the outdoor space (Zimmerman and Land 2014, Kamarainen et al. 2015). For reasons of practicality and cost, this article focuses on AR experiences that are accessible through mobile devices (like smartphones and tablets), rather than those that require a head-mounted display. We present examples that use a combination of location-based and vision-based triggers. As described below, the affordances of mobile and location-based AR align most closely with learning goals relevant to ecology and environmental science. Ecologists and ecosystem scientists bring to bear sophisticated conceptual models and background knowledge when they observe or study natural systems (Eberbach and Crowley 2009; Kamarainen and Grotzer, in review). These perspectives provide scientists with a "search image" that can help them identify patterns, notice things that are unusual, pay attention to relevant signals, and connect their observations to prior understanding of natural history. This leads to the question: "How might AR support learners in seeing the world through an ecologist's eyes?" While AR platforms and experiences have only recently reached a level of maturity that makes broad use in learning contexts feasible, research using early versions and prototypes provides compelling arguments that well-designed AR experiences support learning. (A short review of this active area of research is provided below.) Augmented reality can support ecology learning through revealing hidden or invisible aspects of the system, and linking visualization of hidden processes with macro-scale or emergent outcomes. Dunleavy (2014) refers to this as using AR to "see the unseen" and highlights this as a general principle for designing impactful AR for learning. Prior research supports the idea that prompting students to notice and reflect upon processes that are responsible for patterns and change in natural systems can support shifts in student thinking from static or event-based notions of causality toward process-based explanations (Lindgren and Moshell 2011, Grotzer et al. 2013). There are a number of ways AR can be leveraged to support these outcomes. Augmented reality locations can be positioned at places where one wants students to observe an object, pattern, or phenomenon that they might not otherwise notice—for example, a nesting cavity created by a woodpecker, the layer of silt left behind after a spring flood, or a path cut through brush by a deer. As described by Eberbach and Crowley (2009), engaging in scientific observation is a more challenging skill than is generally appreciated, and requires the coordination of disciplinary knowledge, practices of observation, and the application of one's attention. Novices often do not know what to look for, so may quickly give up or overlook interesting artifacts. Augmented reality can be used to alert students to an opportunity to observe something meaningful that connects with ideas they have been learning; and tips, reminders, and reflective prompts embedded in the experience can encourage students to adopt practices of observation that mirror those used by experts (Klopfer and Squire 2007, Dunleavy and Dede 2014, Grotzer et al. 2015). Augmented reality can also be used to overlay or link multiple representations of a system or phenomenon (Zimmerman and Land 2014, Kamarainen et al. 2016). Many ecological changes are driven by organisms or processes that are too small to see, while emergent patterns or outcomes may be best visualized from a bird's-eye perspective. When making sense of scientific visualizations and abstractions, students benefit from viewing multiple forms of representation (Ainsworth 1999, Wu and Shah 2004), manipulating and interacting with physical models as well as visual representations, engaging in metacognition and reflection related to the visualizations (Chang et al. 2009, Wu and Shah 2004), and making links among representations (Wu and Shah 2004). Augmented reality can support this by juxtaposing multiple representations or overlaying them with a physical pattern or phenomenon, which allows the user to easily connect and compare the representations without having to hold one in mind while accessing a second (Tang et al. 2003, Pathomaree and Charoenseang 2005, Radu 2014). Augmented reality visualizations can be used to communicate changes over time by embedding views or narratives that describe the history of a place (Zimmerman and Land 2014, Grotzer 2015), or by pointing to evidence of change over time that may be present within the landscape. Ecosystems can have long "memories"—the abiotic conditions, species composition, and relationships present may depend on what happened at the site last season or long ago. Through repeated visits and careful observation, ecologists often develop deep knowledge of the history of a place, its rhythms, and its phenology. Using AR to provide visitors with time lapse, before-and-after, or "what if it were winter" views of a space that they may only visit once can provide a powerful shift in perspective. For example, residents near a community park that was the location of one of our augmented field trips had fortuitously installed a hidden video camera and collected footage of nocturnal and diurnal visitors to the park. We embedded a highlight reel from the camera in the AR experience and activated the video when students arrived at the tree in which the camera had been mounted. Students watched as the nighttime footage showed a fox passing through, a pair of raccoons climbing the tree, and a coyote urinating near the base of the tree to mark its territory. The next clip showed a daytime scene of a neighborhood dog sniffing the location and adding his own scent to the mix. The AR experience prompted students to consider that there are many organisms that frequent the park even though they may not be seen during the daytime field trip. These ways of making the invisible visible help students to see an environment through an ecologist's eyes. How complex is developing these kinds of learning experiences? The basic architecture of location-based or vision-based AR platforms includes two parts: (1) an online editor that the designer uses to upload media, link those media with GPS locations or visual QR codes, and orchestrate the sequence of events the user will engage with during the experience; and (2) a client-side application downloadable to a mobile device that allows the user to login and access the experience the designer has constructed. There are a number of location-based AR platforms that allow you to design your experience for free, and most also allow you to share the experience with unlimited users for free (e.g., ARIS, TaleBlazer, Aurasma), though some require a fee to use the experience with more than one user (e.g., FreshAiR) (see a list of experiences and platforms in Dunleavy and Dede 2014). A number of these platforms have recently been applied to undergraduate learning contexts. Klopfer and Squire (2008) summarize how a predecessor to TaleBlazer was used to create a mobile AR game called Environmental Detectives that supported undergraduates in an environmental science course. More recently, work by Clements (2017) outlines how the TaleBlazer AR platform was used to design a guided tour of a canyon for an undergraduate Physical Sciences course. Meanwhile, Holden and Sykes (2011) offer an example of the use of ARIS to support place-based foreign language learning by undergraduate students. This body of work contains valuable guidance for the design of location-based AR activities for undergraduates. Each location-based AR platform outlined above has pros and cons, but they share similar design features that enable a designer to embed media and link those media with location-based or vision-based triggers. A designer can upload different forms of media (including text, images, audio, and video) into the online editor and then embed these in the experience by linking them with particular locations or "characters" (e.g., a simulated Ranger) who are part of the narrative of the experience (Fig. 2). Locks and conditions can be used to turn locations and media on and off so that, for example, the user must stop at the "toolbox" and pick up a measurement device before they will see a sampling location on their display. In its simplest form, the experience may unfold as a series of sequential stops on a virtual tour of the location—akin to a tour guided by a virtual naturalist. But the platform also allows a designer to thoughtfully use locks, items, and triggers to develop creative narratives, compelling games, and interactive social experiences that engage learners in new and unexpected ways (J. Reilly and C. Dede, forthcoming). Once the design is complete, the experience must be published; then, it becomes accessible to the user through the application running on their mobile device (see Fig. 3 for an example of the user interface). A number of freely available platforms provide off-the-shelf experiences by allowing a designer to copy or drag-and-drop an experience from one location to another. This is a quick and easy way to get started, but it is always important to consider the relationship between the experience and the place that it will be enacted. Some AR experiences are designed in a way that is place-agnostic—where the virtual aspects of the experience refer to fairly ubiquitous environmental features (like air, water, or trees) such that the experience may be transported to a new location without losing the fidelity of the learning experience. In contrast, other experiences are designed to be deeply place-dependent, referring to specific and unique characteristics of the environment that may not be found in other locations (e.g., a snakeskin discovered on the ground; Kamarainen et al. 2015). When using off-the-shelf experiences, it is critical to consider whether and how the original design of the experience aligns with the features of your location. Modifying an existing experience is another way to get started. Modifying a functioning AR experience can allow you to observe how the experience appears in both the design and user interfaces in order to understand the relationship among the components. Once you have unpacked the existing design, you can begin to bring your own ideas into the mix by uploading and replacing media, changing the text, adding new locations, or changing the order and sequence of user interactions. Each of the AR platforms mentioned above (ARIS, TaleBlazer, FreshAiR) has online resources, manuals, and tutorials available to support new designers. These resources make it fairly straightforward to modify a template or design and build your own AR experience from scratch. In addition, an exciting way to use AR in your undergraduate classroom would be to engage your students in the design of their own AR experiences. Scholars suggest that engaging students in design is a powerful way to support learning because it engages students in "learning by doing" and prompts them to incorporate multiple tools and resources, the design process can guide learners into a way of thinking, and frustrations that arise during the design process become rich opportunities for problem-solving (Squire and Jan 2007, Petrich et al. 2013). Prior work outlines benefits of engaging students, as young as middle school, in AR design activities including critical analysis, deeper thinking, and increased understanding of issues in their community (Klopfer and Sheldon 2010, Coulter et al. 2012, Bower et al. 2014, Martin, Dikkers, Squire, and Gagnon 2014). Given the diversity of freely available AR platforms, and their ease of use, the design of AR experiences may be an ideal way to engage undergraduate learners in deeper understanding of the environments around them. You may imagine a course project in which students are required to collect images, video, or audio that captures and represents how different ecosystems on campus change over time (perhaps over the course of a day, a month, or a season). They might embed media they have collected in an AR experience that can be shared with the rest of the class or even the campus As students use and the AR experiences created by their they might understanding of diurnal patterns, or and more deeply how ecological processes and in the ecosystems that them. was a research project by the which focused on the utility of AR experiences with virtual environments for middle ecosystem science Through a process of we designed and a number of activities that focused on different physical and technological et al. 2013, Grotzer et al. Kamarainen et al. 2016). this process of we used the ARIS platform 2010, and to a number of location-based AR experiences with associated that provide a of of experiences from to We have these available and that will and modify them to best with the unique features of their own the design features we provide a of a of the experiences that are is designed to help students understand the of in with a on the of of and the processes of and that middle students to the of and to make sense of processes that are not visible in which to in about and et al. and in understanding these processes into undergraduate classrooms, by of the relationship between and and these can be to et al. et al. such the students to a or through their environment. The as part of a and students the of through by or running off of the through and an is from a The as part of a within a and after being by a virtual the material to the of the and is used by The is and makes its way into the and this is up by a and part in with the to being a part of a in a different The AR provides ways to engage students in active and learning that have been shown to be in student learning (Price and Rogers 2004). Students around the environment the of water, and Physical of the of students a more of material (Price and Rogers 2004, and Moshell Through engaging students in this and the design to give a sense that and though not are all around as well as a physical sense of the for of in the environment. Students are prompted to look for and notice features of the environment (e.g., look for notice and of and that connect with the of their virtual (Fig. A QR code to the of a tree an of by of and to and within a (Fig. Through of multiple and with real objects can and otherwise hidden processes including and the through multiple that the in different students into of may change to and become a part of different physical but it is (Fig. The includes classroom activities that the ideas that are through the augmented field experience. In students might build their own of the or on their experience. We used the outlined by and to help students about the and of in the of and The provides a way of the of in an thinking about a and where it is found over it might from one place to it might or in the original location and in its original or it may or be from one to another through a physical, or The design of is that the experience can be used in place that has water, and some We to use representations of and which are compared to current understanding of but these representations are easy for students to and the experience. The experience students through a for and the number and diversity of shown is this experience not represent the and patterns of and that characterize and is to reveal hidden processes in the and of in in on these learning it aspects related to the of while One potential might be to engage students in designing that be into the including that unfold over different time or related to different (e.g., or engaging students in of the they be prompted to consider ways in which the original experience represents the and of and in what ways it be to represent what is currently about material in powerful of this would be to both and through the processes that are in the current of This would help in undergraduate learners related to understanding how is used and during basic ecological processes and et al. The is as a of a that students in a virtual world called et al. 2013). In the students a virtual a and collect information and evidence to build an about what happened to the The experience students to a real or to whether a is to in the real The AR leads students to visit a measurement "toolbox" to pick up environmental might include or (Fig. in students the area and collect on the of a or (Fig. Through prompts embedded in the AR they share and compare their to those collected by their and collect other evidence about that might their The of this experience is to support students in their own data, and a that the and of natural The location-based AR triggers guide students to a location where they can pick up a that will allow them to of their environment. In to students the space and resources, the AR experience and about how to use the This support can allow students to the at their own through the and collect and at location they We have found that students an experience that is designed to offer a between and After having collected from multiple locations around the students are to a where they their on and them to number and (Fig. As more students visit the patterns and students can see in the of or can notice spatial patterns in the on the multiple representations of the can help the students make and connect the in with physical features of the environment. In the part of the we use the affordances of AR to help students engage in deeper interpretation of their They have to visit of the locations that is interesting to them and can engage with each of the The measurement experience a number of learning goals that are relevant to undergraduate Students engage in to scientific as the experience students to use environmental to collect their own These are then displayed in a visual representation that allows the and of from multiple and In our we used physical visual displays available in the but you might imagine using a to student and make it available for activities can be leveraged to support students in and applying to make sense of the spatial and patterns in these of what makes AR a powerful tool is that it allows the to support student learning of the of the and to do so in ways that connect with concepts or practices in the course. You can use the AR to reflection and metacognition by the user to consider how what they are in the real world connects with what they are learning in These will be most if students are opportunities to engage in reflection and metacognition in other activities during the course. way that these AR activities can be into would be to bring from the field experience into the classroom for or may include data, images, or We found a combination of to the classroom support rich about the behind and also supported students in conceptual models material or et al. Kamarainen et al. 2016). the supports and each experience is different and the diversity of experiences can be leveraged in the classroom to support learning that is and Augmented reality will be most powerful when it is thoughtfully into the of in ways that and connect with knowledge and practices in The experiences were through an design process that and the experiences on with middle students and et al. 2013, 2016). While the that are available are to middle the and language of these experiences be to an Many of the concepts by these activities (e.g., of understanding the making sense of are even for undergraduate students and align with learning goals outlined in the and in for the of 2013). the learning outcomes that are outlined in the using AR with your undergraduate students help you connect the concepts you are in class with outdoor environments that students through Students often evidence that what they are learning about connects to their learning experiences in the places they through each can make the you use AR to give your students so they can see in the tree near the to the they may of it each time they the tree on their way to The idea of including your students in the design process has the potential to support both learning and engagement and Klopfer 2010, Coulter et al. The AR platforms offer rich for creative and students are to up with ways to engage and communicate with one another of that offer a powerful or to the way the ideas are by the The AR platform offers ways to experiences that have been so that the easily review the students' experience to offer on and Augmented reality makes it to design multiple field trips with different to as an not to the associated with a field or time of class to students during the experience. The of undergraduate students have their own so an AR experience at their own on their own even if the location is not on In ecologists and ecosystem scientists with them a of background knowledge when they into natural systems and Grotzer, in and students only access to a small of this through classroom with these and knowledge through the use of AR has the potential to support deeper understanding of concepts in the and more meaningful engagement with the environments around research was supported by and by the was supported by the of of Sciences through and or in this material are those of the and do not reflect the views of the the of or