The User Interface and Tutorial subteam was responsible for creating immersive virtual experiences capable of teaching the necessary steps of operating manual machinery. The primary goals the subteam set out to achieve include:
- Designing an application that accurately replicates the audio and visual experience of drilling on a lathe.
- Creating tutorials capable of teaching new users a drilling operation as effectively as traditional in-person instruction.
- Implementing activities that promote retention of crucial knowledge to complete the drilling operation.
The primary goal of the simulator is to be able to serve as an effective, independent learning tool capable of teaching the processes of operating a lathe. In order to accomplish this, an application was created with the game-engine Unity that consists of three key components: learning, activities, and practice.
Figure 1. VIMS application main menu
With it being necessary for the application to successfully teach a naive user the specifics of a lathe drilling operation, a tutorial was created to help instill declarative and procedural knowledge, both of which are essential to learning. Declarative knowledge is knowing that certain information is true or not true, while procedural knowledge is knowing how to do a specific act. The embedded tutorial fosters these knowledge types in several ways. First, to be sure that the team had a strong understanding of the drilling operation at hand, a point of view video was recorded on a GoPro worn by a Lafayette College lab technician as he completed the procedure step by step. This video was then reviewed, and each key step of the procedure was broken down into a list consisting of 17 steps. With this breakdown, the tutorial was then created with each step consisting of text describing the step, a voiceover explaining the step in more detail, as well as a video played on a loop of the step being completed. Once the voiceover for the current step is complete, users are then able to either continue to the next step or replay the current step to review. Along with this, users can access a part drawing of the piece being manufactured as well as a labeled diagram of the lathe at any point during the tutorial if need be.
Figure 2. VIMS application tutorial scene
Additionally, to further promote the retention of knowledge in the application, quiz activities are embedded throughout the tutorial. Consisting of multiple-choice questions, multiple answer questions, and interactive tasks, these 9 activities allow users to test their knowledge of key components necessary to complete the drilling operation. Questions include lathe part recognition, knowledge of safety practices, procedure outlining, etc. This method of declarative evaluation was recommended by Professor Nees and is consistent with the method of evaluation highlighted in the report Declarative knowledge acquisition in immersive virtual learning environments by Rustin Webster. After submitting their answer, users are then shown the correct answer along with a detailed description of the answer. When complete with the full tutorial, users are then shown their score and a text file including the user’s answers is saved locally to the application’s directory.
Figure 3. VIMS application activity
Finally, to complete the user experience, users are able to practice newly taught skills by interacting with the physical device which controls a virtual environment accurately representing a lathe. To do so, a virtual environment was created that accurately models the Colchester VS-13 Lathe and other aspects of a machine shop necessary for the operation. Referencing the video discussed prior, each component of the lathe was modeled in the 3D computer graphics software Blender, making sure the model was correctly scaled and each of these components were placed in the correct position on the virtual lathe. This model was then imported into the Unity project and combined with particle simulations, to recreate metal chips forming, as well as accurate audio. Scripts were then created in C# to allow for serial communication between the Arduino Nano in the VIMS system and the Unity application. Parameters such as encoder counts were communicated in order to allow for the virtual handwheel and drill bit to move accurately with the physical device. Embedded at appropriate points throughout the tutorial, users are prompted to utilize the VIMS system and virtually complete a center drilling operation as well as the main drilling operation. After calibrating the handwheel so that it correctly lines up with the virtual handwheel, voiceover and pop-up instructions guide the user as they drill into the virtual piece. As they drill, various visual cues pop up indicating that the piece is clogging, clogged, or has been cleared. This is important because it allows inexperienced operators to understand what clogging feels like and how to properly clear the chips. Within this practice scene, users have the ability to switch their virtual view of the lathe, read their depth into the piece using the rulers on the drill chuck, access the part drawing, and restart the operation at any point. Once the drilling operation is complete, users are able to continue with the tutorial.
Figure 4. VIMS application drilling practice scene