Dahl D.A. (Ed.) Multimodal Interaction with W3C Standards: Toward Natural User Interfaces to Everything

Springer International Publishing, Switzerland, 2017. — 430 p. — ISBN: 9783319428147The world of computing is changing rapidly, as we move from desktop computers to tablets, mobile phones, watches, rings, and other things such as devices containing embedded computers or sensors connected to the Internet. New modes of input to computers are evolving from keyboarding and clicking to speaking, gesturing, seeing, and sensing our actions and behaviors. Future applications will integrate information from one or more user inputs and determine the appropriate response.
Standards are one approach to taming the exponential growth of this complexity and opportunity. Standard APIs hide the complexity of how hardware and software platforms work. These APIs enable application developers to access the functions provided by platforms and to create new and exciting applications. Given a standard API, different platform developers will optimize internal processes for specialized hardware or software, enabling application developers to (1) choose the best platform available for their application; (2) implement software on multiple platforms, using the same API; and (3) switch from one platform to another as new platforms become available and old platforms become obsolete.
This book introduces existing and potential standards for multimodal technologies and provides examples of how these standards can integrate multiple modes of input for user interaction with existing applications. It also provides suggestions about the features, functions, and capabilities that new platforms might provide to application developers.Standards
Introduction to the Multimodal Architecture Specification
The Role and Importance of Speech Standards
Extensible Multimodal Annotation for Intelligent Interactive Systems
EmotionML
Introduction to SCXML
Dialogue Act Annotation with the ISO 24617-2 Standard
Six-Layered Model for Multimodal Interaction Systems
WebRTC: Handling Media on the Web
Implementations
Developing Portable Context-Aware Multimodal Applications for Connected Devices Using the W3C Multimodal Architecture
SCXML on Resource Constrained Devices
Standard Portals for Intelligent Services
Applications of the Multimodal Interaction Architecture in Ambient Assisted Living
Applications
Assembling the Jigsaw: How Multiple Open Standards Are Synergistically Combined in the HALEF Multimodal Dialog System
A Case Study of Audio Alignment for Multimedia Language Learning: Applications of SRGS and EMMA
Future Directions
Discovery and Registration: Finding and Integrating Components into Dynamic Systems
Multimodal Interactivity in Foreign Language Testing
Multi-Device Applications Using the Multimodal Architecture
Multimodal Interaction Description Language Based on Data Modeling
Multimodal Fusion and Fission within the W3C MMI Architectural Pattern