Eye on Research: 80Days' Achievements

» Interactive and Adaptive Competence Development
» Interactive and Adaptive Storytelling
» Macro and Micro Adaptive Technology
» Learning Resource Integration
» Game Design and Didactic Design
» Game Development
» Evaluation

Interactive and Adaptive Competence Development

This work package focused on developing a sound psycho-pedagogical framework for educational games by integrating models of adaptive personalized learning with those of adaptive interactive storytelling. This framework is mainly built on the three pillars cognition, motivation, and storytelling. The first pillar cognition seems to be obviously important for game-based learning, because the major purpose of an instructional game should be that it teaches something. Therefore established theories and approaches in cognitive psychology need to be surveyed that are suitable for the application in educational games, and essential implications for the design of these games should be drawn. The same applies to approaches of motivational psychology, since motivational aspects are crucial for effective learning. The last pillar integrates narrative/storytelling in the field of game-based learning, feasible to enhance immersion and engagement of the learner and extensively used in commercial computer games to attract players. Considering these three pillars in the elaboration of a theoretical framework for educational adaptation enables adapting educational games to a specific user in a specific learning situation

Non-invasive Assessment
The very basis of a non-invasive, continuous assessment of learning progress and motivational states is to monitor and interpret the learner’s behavior in the game. To achieve this, we utilize the formal framework of Competence-based Knowledge Space Theory (CbKST). Originating from conventional adaptive and personalized tutoring, this set-theoretic framework allows assumptions about the structure of skills of a domain of knowledge and to link the latent skills with observable behavior. It provides an internal cognition-based logic that is quite similar to the logic of ontologies: well-defined entities (the skills) are in a well-defined relationship (a so-called prerequisite relation). The domain model, the set of meaningful skill states, and the resulting set of meaningful learning paths are combined with a model of tasks and problems within certain parts, so-called learning missions, of the game (equivalent to conventional “learning objects”), the so-called problem space.
In the project we addressed two crucial aspects, on the one hand technical limitations due to a high computational load and, on the other hand, we concentrated on the assessment of motivational states of learners, gamers, and learners in educational gaming situations. The conceptual research work was accompanied by computer simulations to investigate feasibility and correctness of the approach.
A strong focus of our work in the project concerned motivational aspects, in particular an autonomous, non-invasive assessment of motivational states - aiming at the development of a comprehensive higher level adaptive framework usable for intelligent adaptation in educational games and extending the state of the art for adaptive educational technology. Theoretical work focused on finding and systematizing concepts and theories of motivation psychology suitable for the field of game-based learning. This effort aims for the integration of motivational concepts in a broad framework for the development of educational games, and more precisely for finding ways of motivational adaptation.

Adaptive psycho-pedagogical interventions
In addition to assessing the learner’s cognitive or motivational states, appropriate and tailored interventions must be enabled to be autonomously performed by the system. As important as it is to avoid comprising the game’s flow by assessing learning progress or motivational state, are interventions embedded in the game. Interventions are hints, suggestions, warnings, or feedback. We concluded, classified, and systematized the following general types of interventions:

Educational interventions provide the learner with specific information (i.e., skills) if the system concludes that the related skills are lacking. In the game context such interventions can come for example from non-player characters.
Problem solving support and guidance provides the learners with information about their current state in the game-related problem solving process. To give an example, if the system detects that a number of actions did not decrease the distance between the present problem solution state and the target state, the system can trigger a hint that suits the present problem solving state.
Meta-cognitive interventions are supposed to foster reflection about the learner’s own abilities, confidence, or self-esteem. A typical realization of such intervention type is to let a non-player character (NPC) ask specific questions like “are you sure?” or “why did you do that?”.
Assessment interventions are a special form of intervention. If the probabilistic assessment (of either learning progress or motivation) does not lead to clear results after a certain number of actions, the system can trigger interactions to improve the assessment. Typically this can be realized by providing the learner with different problems/tasks or by specific questions through an NPC.

Dissolving interventions are a further form to provide the learner with specific information. The purpose of this intervention type is to provide the learner with the solution of a problem/task if the learner wasn’t able to do so within a reasonable number of actions. Such interventions, ultimately, shall assure that the game can continue even if the learner is not able to solve a problem/task. Of course, for didactical reasons, this intervention type might not be used for all problems/tasks.

Motivational interventions are supposed to retain the learner’s motivation on a high level or to intervene when the system detects that the motivational states (potentially) decreases. Forms of such interventions are feedback, praise, incitation, encouragement, or directing attribution of success or failure (from a motivational point of view the learner should attribute success to his/her own abilities and failure to external components such as bad luck).

All interventions of a game require a manifestation in form of game assets (e.g., a sound file with a specific sentence). Of course, not all possible interventions can be realized. In general, we pursue and propose an approach of using interventions conservatively or sparingly. We are perfectly aware that repeated inadequate interventions due to misinterpretations of a situation (e.g., assuming a lack of motivation on the basis of no actions for longer period of time while the learner just has gone to the toilette) are a significant harm to motivation, engagement, and the game flow. The conditions under which a certain adaptive intervention is given are to be developed on the basis of psycho-pedagogical rules.
The rules for triggering educational interventions and problem solving interventions, for instance, are defined based on cognitive psychological considerations in tight relation to the continuous assessment of skills in terms of CbKST. Through the definition of threshold values on skill probabilities an according intervention is prompted if the non-invasive monitoring procedures of the learner’s actions provide substantial evidence for lacking skills or an increasing distance to the problem solution. Rules for motivational interventions, on the other hand, are developed grounding on motivational psychology, referring e.g. to attribution theory and the ARCS model of motivational design. Correspondingly, continuously unsuccessful behaviour and unconfident reactions arguing for a decrease or lack of confidence and motivation will trigger an intervention fostering motivation and suggesting self-worth enhancing attribution styles.

Interactive and Adaptive Storytelling

The major achievements during the project are situated in the fields of theoretical backgrounds of digital storytelling in adaptive educational games, implementation of methods elaborated as part of the research and the creation of an authoring tool for adaptive educational games.

As a theoretical result, a framework for higher level adaptation was described, integrating the ideas of adaptive digital storytelling with the CbKST-based educational adaptation mechanisms (micro and macro-adaptation).

Closely related yet more focused on the area of storytelling are the results achieved in the context of “Theoretical storytelling framework”. Here, research efforts into foundations of digital storytelling and the creation of a format for the description and execution of digital stories were concentrated. One of the results of this work was the recommendation to use the “Hero’s Journey” story model as an established and formalized pattern for narrative which nevertheless was adaptable for use in educational games. This choice highlights the connection of this task with didactic and game design tasks in the project, as choices in one area influenced the other areas.

For encoding educational game narratives, the format ICML, created as a result of previous projects, was extended to be used as the basic format of the 80Days game framework. One major extension was the introduction of the concept of Narrative Game-Based Learning Objects (NGLOB). This refers to story or game parts which are annotated with information about their narrative function as well as information about their suitability for certain players as identified by their player type and information about the objects’ learning function, indicating which skills are required for a certain part of the game and which are taught to the player during this part of the game. This general and reusable set of metadata can be used to annotate game objects in various contexts and across different games and genres.

Based on the theoretical results achieved, the proposed methods were implemented into the “Narration Engine” component. This software component, being used in the 80Days technical framework and therefore the demonstrator game, is able to parse ICML files with the NGLOB-based annotations and to execute the game based on the user. In order to react and adapt to different users, the Narration Engine is synchronized with the Adaptive Learning Engine. The Narration Engine offers a balance of control between the player and the author, which is a common challenge in digital storytelling (referred to as the “narrative paradox”). This is achieved by setting up parts of the game in such a way that the author’s vision of the story is enforced while other parts can be freely navigated by the player. In this case, the game is controlled based on the parameters for adaptivity the author has set up during the authoring phase of the game’s story. For actually executing the story, the Narration Engine is also integrated with the Game Engine displaying the game to the player and accepting inputs.

In order to properly demonstrate the effects of the 80Days adaptive framework and the Narration Engine, a second line of demonstrators under the name of “Bat Cave” was created mainly by TUD. This demonstrator did not feature a 3D game engine as the other demonstrator games, but only a relatively simple 2D environment. However the other components of the 80Days framework such as the adaptive learning engine are included in Bat Cave, which can therefore be used to evaluate the approach of 80Days in a more controlled and transparent environment. Using Bat Cave, various information usually hidden from the player by design can be visualized, allowing the tool to be used for evaluation as well as rapid prototyping due to being able to check the effects of setting various parameters on the adaptive algorithms included in the adaptive framework.

The visual editing component for the game’s story was developed during the project as a tool for allowing authors to create the story for an adaptive digital educational game without the need to edit individual files or write statements in a programming language. Connected to the change of partners from ZGDV to TUD, it was decided to complement the Story Editor with a complete authoring tool, which allows the creation of entire games by non-programmers. This tool, called “StoryTec”, includes the Story Editor as originally envisioned as one component. Using StoryTec, an author can create and organize the structure of the game’s narrative, integrate content such as images or videos, set up interactive behaviour (for example directing a game character to deliver a certain line of dialogue when an action in the game was carried out) and configure parameters for adaptivity by means of the NGLOB data described above.

Macro and Micro Adaptive Technology

80Days has successfully produced an Adaptive Engine that is conceptually abstracted and technically separate from the Story and Game Engines. It successfully implements the skill assessment and motivation assessment logic necessary to infer the learner’s current state. This model is used to adapt, in runtime, the game and story to enhance the educational effectiveness of the game. The Adaptive Engine represents a number of significant advances over the engine developed in ELEKTRA. Namely, the 80Days Adaptive Engine –
>>> successfully implements motivational adaptations. These adaptations stem from the MAE (Motivation Assessment Engine) and the corresponding rules for motivation interventions.
>>> is truly scalable. There were concerns about the scalability of the CbKST approach and through intelligent encoding, partitioning and reduction in the knowledge space the Adaptive Engine can now scale successfully across a large number of learning situations. In ELEKTRA this was limited to less than five LeS.
>>> can integrate successfully in a micro and macro adaptation workflow. It can cooperate with the Story Engine to ensure the storyline can grow successfully to accommodate additional learning objectives.

Learning Resource Integration

In an initial phase of the project the focus has been put on the identification and description of resources to be used in the generation of terrain. In particular several different terrain elevation datasets as well as satellite images have been downloaded and evaluated with the purpose of creating a terrain model. One of the first objectives of modern geography teaching programs is to provide a geographic framework to the pupils allowing the spatial localisation of the different geographic phenomena and aspects constituting the main topic of the geography class. In 80Days we tried to create this framework by setting the action of the game in a terrain representing western and central Europe (in this case this is the framework). From this point of view, the geographic datasets selected and utilized in the first demonstrator constitute not only a resource from a ‘technical’ side, but also an important learning content. Furthermore the work of selection and description of learning resources led to the definition of a clearly arranged repository structure. Resources have been classified and stored due to their content following a directories structure consisting of three main categories (Basic Material, Topographic and Thematic Resources, Additional Resources) each of them composed by several sub-categories. At the same time for each identified resources a metadata sheet has been compiled. These sheets collect information about dataset format, provider institution, coverage, resolution, IPR, etc. Beside the identification of specific geographic datasets necessary for the rendering of the 3D environment, attention has been also put on additional resources such as 3D objects, videos, maps or pictures. Through the investigation of the availability of copyright free 3D objects has been possible to increase the visual appealing of the 3D environment by placing them on the generated terrain surface. By identifying utilizable topographic and thematic maps it has been possible to reduce the work necessary for the production of navigation elements thanks to the utilization of already existing and suitable material (maps). Videos and pictures have been identified used for the illustration and completion of certain learning situation avoiding the manual creation of specific images or animated sequences.
Another task focused initially on the integration of the different datasets identified and selected through and on their utilization for the generation of an immersive game environment using computer game technology. On the one side the main integration and harmonisation problems have been identified and described. On the other side investigation has been carried out in order to allow the development of tools for the management of the selected data.. Projecter has been specifically implemented in order to allow the conversion from geographic datasets from typical GIS formats into something utilizable by the game engines in the process of terrain generation. In particular harmonisation in terms of coordinate systems and spatial resolution is carried out. At the same possible the analysis of the functionalities of GIS software allowed the definition of concrete steps leading to the integration of data in both raster and vector format in the same final result. It has been also possible to demonstrate the possibility to use the same approach for rendering other regions at different spatial resolution without having to invest particular work on the adaptation. Further on research work and tests have been carried out in order to implement the utilization of 3D objects in the rendered environment. An approach allowing a semi-automatic placement of 3D object according to rules based on the information contained in land cover geographic datasets has been developed. Placement of objects corresponding to defined coordinate pairs is of course also possible.
The combination of the results of the various tasks resulted in the development of the final resource integration methodology. The methodology is built on the utilization of freely available digital resource for the reduction of DEG’s development costs. On the one side freely utilizable geographic datasets can be used, combined and integrated into a computer game engine in order to generate a 3D navigable virtual environment. On the other side the methodology describes the principle allowing the selection and further utilization of additional external material such as videos, pictures or map in form of assets. The final resource integration methodology presents step by step the operations necessary in order to achieve the mentioned results. Thanks to principles of the resource selection process, description of GIS harmonisation functions and explanation of the tools specifically developed for supporting the framework also an external researcher not involved in 80Days should be able to re-use the developed approach. Beside the definition and detailed description of the single steps necessary for the re-utilization of the resource integration approach developed in 80Days, ETHZ invested particular efforts in the development of Terrain Viewer, a software allowing the generation of virtual environments based on geographic datasets. This program allow the generation of 3D virtual environments outside of this specific DEG context. Terrain Viewer is a stand-alone program utilizable independently from the 80Days DEG context (Lizard, Atlexicon, BatCave). This will on the one side allow computer game experts to easily generate virtual environments and game assets. On the other side the methodology offers researcher in the field of cartography and geovisualization to experiment a potentially interesting new technology for the appealing representation of geographic information.

Game Design and Didactic Design

In the course of the project two strands of demonstrators evolved: “Lizard” and “Bat Cave”. Three versions of Lizard were designed and developed. Based on the user validation results of the respective forerunner version the demonstrators improved iteratively their learning game design and the research concept of Micro adaptivity and story pacing in particular. The demonstrator “Bat Cave” was designed as a mock-up system with a massive but simple content pool in order to test the macro adaptive technology and enable the realization of several learning and story paths.

The design team produced three Learning Game Design Documents for “Lizard”. The achievements in the Learning Game Design for Lizard was improved by user validation tests in classrooms, interviews with focus groups, participatory design sessions at the 3rd YOUTH FORUM 2008 and the GAMESCOM 2009 in Cologne. The principles of good learning game design of the forerunner project ELEKTRA were developed further: On the one hand we developed new didactic design and game design (game metaphor, game concept) that suited to the learner requirements and objectives of Geography, on the other hand we gained more abstract design experience on a methodological level because the subject was different compared to physics in ELEKTRA. This was important in order to understand which design methods are valid for at least two very disparate subjects like Physics and Geography and which design elements are subject to the specific requirements of the subject didactics.










  Game Development

We have developed and enhanced main software modules constituting the 80Days Game Engine: scripting engine, physics engine, sound system, GUI system and input/output modules. Great attention has been put to the relationship with other 80Days main software modules, namely Story Engine, Story Editor and Learning Engine.
TCD, responsible for the development of Learning Engine, TUD, responsible for the development of Story Engine and Story Editor and TL responsible for the development of Game Engine, have worked together to define and implement the communication modules between the three engines, fundamental technological step toward the creation of the base technology of 80Days.

In particular the fundamental functioning of the LE (in accordance with the four-stage ALIGN approach) remains the same, but discrete alterations have been made to accommodate adaptive storytelling. Specifically, this involved enhancing and refining the communications protocol to include the SE, the inclusion of a story state database and story context filter and the support of a broader range of adaptation types. The adaptation rules (micro) necessary for the micro missions have also been implemented as well as the automatic extraction of ontology partitions from a global ontology.

On the basis of the underlying technological foundations, game development of the demonstrator strands Lizard and
Bat Cave required:
• to implement design requirements
• to integrate a Text to Speech technology in 80Days Game Engine
• to implement the Terraforming simulation
• improve integration of Game Engine, Learning Engine and Story Engine
• to allow parts of the demonstrator being managed by different engines but keeping real time performances
• to improve terrain rendering algorithm for displaying 3D objects and enhancing visual quality of the rendered terrain
• to improve characters behaviour and animations
• to integrate the character engine with the Text to Speech technology
• to integrate 3D and 2D graphics, sound effects and background music
• to integrate additional external learning resources
• to enable players having a satisfactory learning/gaming experience

T he evaluation studies undertaken in the three phases of the project have been very successful in terms of improving the game prototype (one of the primary goals of the evaluation studies), gaining insights into the factors contributing to the success of adaptive DEGs, and identifying potential research topics to be pursued in the future. Specifically, the close collaborations between the design team, the development team and the evaluation team have resulted in some significant improvements of the game prototype. Amongst others, the increasing stability of the prototype from Lizard 1.0 to Lizard 3.0 is notable. So is its usability. In particular, specific research topics on game-based learning include the role of gender, evaluation of adaptivity, tracking of user experience over time, co-experience in social gameplay, theoretical frameworks for understanding gaming interactions (Law & Sun, under review), are worthy of further research efforts.

Another significant progress we have achieved is the use of eye-tracking technique to study visual attention during gameplay. The enthusiasm over this topic has led to an interesting experimental study conducted under the collaboration between ULEIC and UniGraz. Results are published in various quality papers. Indeed, the recent rapid development of eye-tracking devices has rendered their deployment much easier and user-friendly (e.g. little effort for calibration and large degree of head movement). More important is that the technique has become non-intrusive and much more accurate. Nonetheless, interpretations of visual data such as fixation duration, fixation count and scanpath remain malleable. This issue has been experienced in the evaluation of Lizard 3.0 where eye-tracker was used in some testing sessions. The related findings stimulate us to further investigate it.

The project conducted also two follow-up evaluation studies of the StoryTec authoring tool, which included preparing a questionnaire and test setting as well as carrying out the tests. The first evaluation study was carried out with 26 subjects who tested authoring of a small game in StoryTec using the Think-Aloud method, resulting in useful protocols, and who afterwards filled out a questionnaire. The second evaluation study was carried out with three employees of a German educational game studio, who intensely tested StoryTec and gave feedback on how they could envision StoryTec in their work.