top of page

Mixed-Reality in Classrooms

How to design and offer mixed-reality learning experiences in K-12 classroom settings?

Augmented Reality (AR) can provide authentic learning experiences for students, particularly in geometry learning, a pillar of mathematics. Then, how can we design AR learning experiences to enhance student's learning? How can we integrate this innovative teaching approach into classroom contexts? How can we capture students' learning processes within their AR learning experiences?

 

To answer these questions, I have been actively working on developing AR educational software using Unity3D, and exploring students' learning processes with a lens of learning analytics.

Keywords: Augmented Reality, Embodied Learning, Learning analytics, Geometry learning, Mathematics

Marker-less AR for Learning Geometry

 

AR has much pedagogical potential in learning 3D geometric shapes. In educational research, we commonly have used image-target AR. To use image-target AR, we need to prepare a printed image, and face mobile devices toward it, and then virtual objects pop up on the screen. One of the limitations of image-target AR is that it could lose contexts and limit students' interaction only toward the printed image. However, marker-less AR, a relatively new AR technique, doesn’t need printed images. Instead, it recognizes the environment and detects plane surface areas such as the top of a desk, or a classroom floor. So, the marker-less AR allows students to use their environments for their learning activities without losing context. The video clips below are from a mobile software that I made with Unity 3D + AR foundation. The plane area in the video is a kitchen table in my apartment. After the AR system scans the kitchen table, it recognizes the shape and location of the scanned plane. Students can place 3D shapes or build their own shapes on the scanned plane, and can manipulate the 3D shapes freely. Even though students move their bodies holding mobile devices, the AR system keeps rendering 3d shapes on the scanned plane, as if in the real world.

Marker-less AR for Computer Science Education

 

Marker-less AR could improve students’ motivation and engagement in computer science education. In the game, students are requested to program a 3D car to move to the destination. Tracks and destinations are automatically generated based on the real-world environments scanned through Marker-less AR. The biggest educational potential of this approach is that the learning can be situated in the context as students can use physical objects or environments for their learning.

For instance, I scanned and used three plane areas in my apartment, a small and big table, and a chair. The number of detected plane areas, their size and boundaries, and locations and distances between them are taken into consideration for generating tracks. After generating tracks, a 3D car is located on the starting point, and students can build the car movement actions using those cute icons to move the 3D car to the destination.

Webcam-based Computer Vision for Learning Geometry

 

Abstract: Embodied learning has been considered as an effective way of teaching and learning, having pedagogical potentials. Motion-tracking devices have been frequently used in research; however, it was difficult to apply them in classrooms. To bring embodied learning interventions into K-12 classroom settings, this research used machine learning enhanced computer vision techniques. Three hand gestures that are expected to be mathematically relevant were designed and implemented in a geometry math game through the techniques.

*Presented at ISLS 2021 as a poster, "Embodied Learning in Classrooms: Machine-learning-enhanced Computer Vision for Geometry Math Game."

Gamification and AR with learning 3D building blocks

 
 
 
 

This is an AR math app helping students to develop spatial reasoning skills. Based on building blocks learning activities in the national mathematics curriculum of South Korea, The app provides three functions, and it encourages students to mentally rotate 3D objects and convert 3D objects to 2D or vice versa.

bottom of page