IRB approval was granted for our study to evaluate the effectiveness of VIMS as a learning tool. To test the pedagogical effectiveness of our simulator, two groups were assessed. The experimental group learned how to operate a lathe using VIMS while the control group was taught by a shop technician as ME 210 students traditionally are. When the groups finished their instructional sessions, they went to the lathe for a transfer test where they demonstrated what they learned by performing the procedures they were taught. During this transfer test, data collectors assessed the users’ ability to operate the lathe using a Task Analysis Rubric.
Task Analysis Rubric Used During Testing
| Scale | 0 | 1 | 2 | |
| For 0-2 scale: | No knowledge of procedure | Partial knowledge of procedure | Full knowledge of procedure | |
| For 0-1 scale: | Not Complete | Complete | ||
| Step | Verbal Score | Action Score | Range | |
| Safety Check (prompted) | 0-1 | |||
| Lock center drill into drill chuck | 0-2 | ***If participant omits verbal component but performs step correctly, full credit given | ||
| Slide tailstock into position | 0-2 | |||
| Lock tailstock | 0-1 | |||
| Turn on spindle | 0-2 | |||
| Apply cutting oil | 0-1 | |||
| Make initial dimple | 0-2 | |||
| Turn off spindle | 0-1 | |||
| Slide tailstock back | 0-2 | |||
| Remove center drill | 0-2 | |||
| Insert main drill bit | 0-2 | |||
| Slide tailstock into position | 0-2 | |||
| Lock tailstock | 0-1 | |||
| Apply cutting oil | 0-1 | |||
| Turn on spindle | 0-1 | |||
| DRILLING | ||||
| Grab correct handwheel | 0-1 | |||
| Hand placement on handwheel correct | 0-1 | |||
| Clear chips periodically without prompt | 0-2 | |||
| Drill to desired depth | 0-1 |
The performance data from 8 survey tests consisting of 4 control groups and 4 experimental groups was analyzed by performing two-sample t-Tests assuming equal variance. Because of the inability to reject the null hypothesis in the t-Test, there is no evidence to suggest that VIMS is less effective than 1-on-1 instruction from a professional.
The mean scores of each individual procedure were compared between the control and experimental groups. As indicated by the darker bars, there were only two procedures where individuals from the experimental group scored lower than the control group.
While this data is not statistically significant due to the small sample size, it is a promising starting point suggesting that VIMS could definitely be as effective as standard machining instructional methods. We aim to continue these studies to obtain statistically significant results as the team is further pursuing this project over the summer as a part of the Bergh Summer Accelerator program. We plan to improve the areas where VIMS learners performed worse than those who learned via the instruction of a lab technician. This will include adding a greater emphasis on the importance of these procedures by adding additional assessment questions and tutorial explanations.