Classroom Augmented Reality with Your Smartphone

In this project, we focused on identifying drawings of objects that make up a physical (mechanical) system to create an application that would resemble a simulation of this system drawn on a blackboard in a classroom. Through this project, we hoped to generate motivation and interest among students studying physics. Our main goal was to identify objects in a wide variety of images.
In the beginning, we focused on a preliminary solution of image pre-processing, extraction of strokes from the shapes in the picture, joining of strokes belonging to the same form and the identification of the shapes. We succeeded in separating the objects into strokes but chose to abandon this solution (before the merging phase) and focus on separating and identifying the objects in other ways.
We tried to identify objects by two methods:
1. Identifying shapes based on geometric characteristics
2. Using ‘Bag of Features’.
Using the first method, we successfully identified circles, triangles and vertical rectangles, while in the second method we reached 89.5% success rate in identifying objects.
We also tried to separate the complex system into individual objects. We were able to find many bounding boxes that bound all objects in the physical system image. However, we were not able to select bounding boxes suitable for separating objects in the image.
In summary, we were able to identify separated objects well but we were unable to cope with the complexity of the given system and with the fact that its objects often touch each other. In order to identify the objects, we must first separate them.

Classroom Augmented Reality with Your Smartphone
Classroom Augmented Reality with Your Smartphone
Classroom Augmented Reality with Your Smartphone