Multimodal interaction with semantic Web services and knowledge sources

Cirius is a multimodal interactive system that enables access to information, multimedia content, and services from a semantic database by means of spoken language and gesture. The demonstrator uses semantically annotated content about Konrad Zuse’s computer and related events, organizations, people and shows how it can be called up from a PHEX multitouch console or a smartphone.

Cirius was developed under the terms of the THESEUS research program. The Cirius system handles natural spoken input to answer questions about German federal states, cities, rivers and seas as well as facts about the European neighbouring countries like their area and population.

The system interprets various input modes like natural language in the spoken form and pointing gestures and in addition enables a rapid access to semantic services as well as the ontological representation of the extracted information. These services create access to an unrestricted number of heterogeneous online information sources. Cirius uses Web services to find additional photos, texts, and videos. Documents in the results that have been provided with location data can immediately be displayed on a map.

The core of this multimodal system consists of an ontology-based interactive voice system platform (Ontology-based Dialog Platform – ODP), which was developed at DFKI under the terms of the THESEUS program (CTC-WP4). Cirius Demonstrator – Multimodal interaction with Semantic Services and Knowledge Sources.

The open ODP architecture insures flexible integration and an interface to modality-specific and user side system components in addition to application-specific implementation functionalities. ODP provides especially powerful programming interfaces for dialog-specific processing components, which support an efficient data representation of knowledge structures and their rule-driven processing. The ODP solution is marketed by DFKI spin-off SemVox GmbH.

THESEUS is a research program initiated and funded by the Federal Ministry of Economics and Technology (BMWi) with the aim of developing a new Internetbased knowledge infrastructure, in order to better utilize
and exploit the knowledge in the Internet. Its focus is the semantic technologies being developed in the Core Technology Cluster (CTC) in order to apply and test them in specific application scenarios in areas such as
medicine, business web services, business process optimization.