TECHNOLOGY AREAS: Information Systems, Human Systems

OBJECTIVE: High-level abstractions for new tools and languages capable of increasing the efficiency of expert and student model development for intelligent tutoring systems.

DESCRIPTION: One of the success stories for artificial intelligence and cognitive modeling techniques has been in the area of intelligent tutoring systems (ITS). ITS have proven to increase levels of student learning by 1.5 standard deviations over traditional forms of educations and they have the potential for not only increasing student learning but also increasing instructor domain expertise. ITS applications automate portions of the educational process, including curriculum delivery and adaptation, assessment of student performance and knowledge, and student-oriented guided interaction. Intelligent tutoring systems have been developed across a wide range of domains, from high school mathematics and physics (Anderson, Corbett et al. 1995; Vanlehn, Lynch et al. 2005), to complex procedural skills such as electronics troubleshooting (Katz, Lesgold et al. 1998) to more "ill-defined" domains such as learning to interact with someone from a different culture (Lane and Johnson 2008).

A common approach to the successful operation of intelligent tutoring systems is to model the learning domain itself. The learning domain includes both the task being tutored and the student�s progress and potential pitfalls in the learning domain (Woolf 2008). Task models can be used to evaluate student performance, allowing the ITS to identify errors in the student�s problem-solving actions, the student�s current level of subject mastery, as well as specific gaps in the student�s problem-solving knowledge. By modeling the larger learning domain, the model also gives an ITS a capability to recognize a wide range of errors and incorrect behaviors that, individual students might exhibit, and to tie those errors directly to a known set of knowledge gaps or incorrect strategies that can then be targeted during subsequent lessons.

Although intelligent tutoring systems have proven remarkably successful, their large-scale development and application remain elusive. They are labor intensive and thus expensive to build. Currently, the learning domain models in a tutoring system are almost always developed by cognitive scientists and cognitive engineers, who have extensive expertise in building cognitive models and tuning them appropriately for use in intelligent tutoring systems. While this approach creates high-quality tutoring systems, it is difficult and expensive for instructors to extend existing tutoring systems to new problems types, curricula, error types, reasoning strategies, etc. In an era of rapidly changing missions and training requirements to support them, the cost and inflexibility of "expert-created" models limits the applicability of intelligent tutoring systems in many Navy training areas.

One approach to helping make ITS more scalable and flexible, would be to enable curriculum developers, instructors and less highly-skilled developers to develop ITS models. ONR has invested successfully in improving the cost effectiveness in cognitive modeling via the development of high-level modeling abstractions and task-specific tools (e.g., see Ritter, Haynes et al. 2006 for a review of numerous relevant research projects). For example, the development of programming language or a set of tools that directly support the creation of student models. This approach is in contrast to current state of art, where models are developed conceptually but then must be translated and mapped into an implementation language (like Java). The high-level abstraction makes it possible for modelers to express modeling concepts directly in the code.

In general these methods enable modeling at the level of "task reasoning" rather than the level of "memory retrieval and deliberation" that is current best supported by cognitive modeling architectures. Higher-level and domain-specific abstractions allow end users to concentrate model design on the target knowledge for the models (and the tutoring systems) without having to worry about specific types of memory management and action-selection regimes.

This research topic seeks approaches to improving the cost effectiveness of task modeling for intelligent tutoring systems, focusing specifically on the adaption of existing techniques or new techniques for supporting high-level and domain-specific languages and tools for designing and building ITS models. Proposed solutions should maintain and support the formal nature of model definition while being general enough to be applicable to various ITS modeling approaches, various modeling framework/architectures, and different ITS learning domains.

PHASE I: Phase I addresses two major goals: (1) Reduce the developmental costs of ITS, and (2) Diminish required expertise of potential developers. The bidders should propose a language and/or set of tools that use high-level abstractions to make it easier and cheaper to build models for intelligent tutoring systems. Designs should identify specific, appropriate abstractions gleaned from analysis of existing intelligent tutoring systems and their models, as well as any studies performed with potential end users. Phase I should outline an evaluation approach that will be used in Phase II to create and demonstrate the effectiveness of the new languages/tools and identify specific metrics for measuring the effectiveness of the tools.

Design efforts in Phase I should demonstrate or provide arguments to show the generality of the approach across existing modeling architectures and ITS platforms.

Finally, Phase I should deliver a plan for implementing, evaluating, and deploying the new languages/tools.

PHASE II: Implement and evaluate the language and/or tools designed in Phase I. Design and implement a modeling environment compatible with one or more existing intelligent tutoring systems, demonstrating a capability for end users/instructors to extend existing tutoring systems cost effectively, without requiring the participation of cognitive engineering experts. Refine, complete, and generalize designs for multiple architectures and tutoring systems.

PHASE III: Deploy the new languages and tools as part of the product packages for one or more intelligent tutoring systems. Evaluate cost effectiveness of user-configurable models and curriculum development using the new technology.

PRIVATE SECTOR COMMERCIAL POTENTIAL/DUAL-USE APPLICATIONS: High-level tools and languages for faster Intelligent Tutoring System (ITS) model development

REFERENCES:
1. Anderson, J. A., A. T. Corbett, et al. (1995). "Cognitive Tutors: Lessons Learned." The Journal of the Learning Sciences 4(2): 167-207.

2. Katz, S., A. Lesgold, et al. (1998). Sherlock 2: An intelligent tutoring system built on the LRDC framework. Facilitating the development and use of interactive learning environments. C. P. Bloom and R. B. Loftin. Mahwah, NJ, Erlbaum.

3. Lane, H. C. and W. L. Johnson (2008). Intelligent Tutoring and Pedagogical Experience Manipulation in Virtual Learning Environments. The PSI Handbook of Virtual Environments for Training and Education. J. Cohn, D. Nicholson and D. Schmorrow. Westport, CT, Praeger Security International. 3.

4. Ritter, F. E., S. R. Haynes, et al. (2006). High-level behavior representation languages revisited. Seventh International Conference on Computational Cognitive Modeling (ICCM-2006). Trieste, Italy.

5. Vanlehn, K., C. Lynch, et al. (2005). "The Andes Physics Tutoring System: Lessons Learned." Int. J. Artif. Intell. Ed. 15(3): 147-204.

6. Woolf, B. P. (2008). Building intelligent interactive tutors: student-centered strategies for revolutionizing e-learning, Morgan Kaufman.

KEYWORDS: Cognitive Architectures, Programming Languages, Knowledge Representation, Human Behavior Models, Intelligent Tutoring Systems

Questions may also be submitted through DoD SBIR/STTR SITIS website.