KIT208 Assignment 3 Report

Introduction

What if you could make text books more interactive and engaging?  This app aims to augment existing images that might exist in a text book to provide additional contextual and interactive elements to a prospective user or reader.  

Description of the application

This application is generally a proof of concept for an education style app, where interactive images can be used to supplement experiences and knowledge gained from a static page in a book.  

In the case of this app, the focus is on military aircraft.  Showing an 3D model, plus ability to show more information about the target aircraft through various interactions available to the user.  The user also has the ability to manipulate the 3D model to view it from different angles.      

Generally speaking, Augmented Reality "offers new learning opportunities, it also creates new challenges for educators", ranging from technological to pedagogical (Wu et al. 2013).  Given these challenges that exist, this app aims to be simple to use, with non complicated interactions available to the user.   

The aim is to provide more information to the user/reader that would otherwise be either very difficult, or impossible in a normal static book scenario.  
In this app this takes two forms, the first being a 3D model that the user is able to move around, this scenario would not be possible in a normal book. 
The second is in the form of text that can be displayed through the AR interface.  While text can of course be displayed in a book, this method allows for more content to be displayed in a smaller area (as the text does not all have to be printed before hand) and it also allows for edits or updates, without having to do a full reprint of a book.  

Interaction Design

Some assumptions need to be made with design of the interactions in this app.  In an ideal scenario, this app would supplement a text book of some description, where the image targets are individual pages in that book.  For prototyping purposes this app instead uses image targets taken from images on the internet.  

Now that being said, the interactions if this was a real app would enable engagement not possible with static pages.  This is shown in two ways, firstly being able to show a 3D model of whatever is currently displayed on the page, this allows the user to rotate and examine the model from other directions than the static image on the page would allow.  

The second way is through the use of contextual text boxes that are displayed to the user, this affords two main advantages, firstly showing more information than could be displayed on a static page and secondly, it allows for any information to be corrected or updated without needing a full reprint of a book.  

There are 3 interactions that provide functionality inside the app.  

The first is a collision detection, between any of the aircraft Image Targets and the dial Image Target.  When this interaction is active, a cylinder attached to the dial changes to green, and the second interaction is enabled.  

When the second interaction is enabled, under the conditions described above, the user has the ability to rotate the aircraft model around, both horizontally and vertically.  This is using the standard interaction of the device, either touch or mouse movement.   

While not really an interaction, I'll mention the final component here, as its original intent was to be an interaction as well.  The original idea was to make a virtual button, or mouse click, which when pressed would activate a brief text description of the currently displayed aircraft.  

However, due to time constraints this has been implemented simply as a trigger on each aircraft image target's 'OnTargetFound()' and 'OnTargetLost()' methods, in other words the only interaction the user performs is choosing the correct image target.  

Technical Development

The application has been built to use Augmented Reality, specifically utilising the Vuforia plugin for Unity, targeting at a webcam for displaying VR elements.  

The interactions of this application revolve around a number of 3D models of military aircraft.  

The first interaction occurs when the user brings any of the aircraft image targets close enough to the dial image target, when this occurs, the two image targets enter a collided state and three things happen:

• Firstly, the cylinder attached to the dial image target turns green, this lets the user know the image targets have collided
• Secondly, the cursor mode is locked to the 'Game' display
• Finally, the mouse is now able to freely rotate the 3D model of the aircraft currently displayed

To end this interaction, the user only needs to move the two image targets away from each other, turning the cylinder back to white, indicating the interaction has stopped.  

The second interaction directly related to the first.  When the first interaction is in an active state, the user is able to use the mouse to rotate the aircraft model around to view it from different angles.  

The final somewhat interactive component is the text displays for each of the aircraft.  In the prototype implementation this is simply a trigger on each aircraft's image target 'OnTargetFound()' and 'OnTargetLost()' methods, where the associated text box is hidden or shown.  

The original intent had been to have this text box triggered by some sort of virtual button or click, which would show and hide the text box.  The second part of that original idea had been to have multiple other text boxes for each aircraft, which would show up when a certain part of the aircraft was interacted with.  For example, if the user clicked on the engine they might be presented with information about the engine in that aircraft.  However, these ideas were not actually implemented due to time constraints.  

Description of 3D Models

Spitfire

3D model of a Supermarine Spitfire.  Used as a display model for the Spitfire Image Target, that the user can interact with in AR.  

A6M5 Zero 

3D model of a Mitsubishi A6M5 Zero, this is used as display model, for the Zero image target.  Used for user interaction in AR.  

B2

3D model of a Northrop B2 Spirit, used as a display model for the B2 image target.  Used for user interaction in AR.  

References

3D Assets:

Mitsubishi A6M5 Zero - https://www.cgtrader.com/free-3d-models/aircraft/military-aircraft/a6m5-japanese...
Supermarine Spitfire - https://assetstore.unity.com/packages/3d/vehicles/air/super-spitfire-53217
Northrop B-2 Spirit - https://assetstore.unity.com/packages/3d/vehicles/air/stealth-bomber-56765

Image Targets:

Mitsubishi A6M5 Zero - https://upload.wikimedia.org/wikipedia/commons/b/b9/A6M3_Zero_N712Z.jpg
Supermarine Spitfire - https://upload.wikimedia.org/wikipedia/commons/5/55/Spitfire_-_Season_Premiere_Airshow_2018_%28cropped%29.jpg
Northrop B-2 Spirit - https://upload.wikimedia.org/wikipedia/commons/e/ec/B-2_Spirit_%28cropped%29.jpg
Dial -https://st.depositphotos.com/1000209/4915/v/450/depositphotos_49156621-stock-illustration-template-of-clock-dial.jpg

Other References:

Mouse rotation of GameObject - https://answers.unity.com/questions/193868/rotate-object-with-with-mouse.html https://www.youtube.com/watch?v=CxI2OBdhLno 

Showing and hiding of GameObject through scripting - https://answers.unity.com/questions/970574/how-do-i-showhide-gameobjects-through...

Text for aircraft information popups:

• https://en.wikipedia.org/wiki/Supermarine_Spitfire
• https://en.wikipedia.org/wiki/Mitsubishi_A6M_Zero
• https://en.wikipedia.org/wiki/Northrop_Grumman_B-2_Spirit

Wu, H-K, Lee, SW-Y, Chang, H-Y & Liang, J-C 2013, ‘Current status, opportunities and challenges of augmented reality in education’, Computers & Education, vol. 62, pp. 41–49.