This chapter details HCI&IM R&D needs that are broadly shared by the HCI&IM CG agencies
that were identified in their workshop presentations (chapter 3) and subsequent fact finding
(chapter 4) and discussion. They are organized into four areas:
• Information Creation, Organization, Access, and Use
• Managing Information as an Asset
• Human-Computer Interaction and Interaction Devices
• Evaluation Methods and Metrics
Successes will help enable robust solutions to meet agency mission requirements and national needs.
a
. Information Creation, Organization, Access, and Use
Many workshop presentations included descriptions of agency needs for very large data and
information repositories that can be used in distributed ways by a variety of on-line
communities. The repositories are primarily to be used to support scientific investigations, but
some agencies also need to support data collection and creation, decision making, and some
provide information to wider populations for other purposes, such as education. To better
support all of these uses, the agencies are both conducting R&D toward identifying what tools
are needed and how to build them, and developing tools for creating, structuring, describing,
using, and interacting with their data and information.
To meet these HCI&IM agency needs, R&D is needed in the following areas:
• Human perceptual, cognitive, and neural processes for obtaining and using information
• The processes that computing systems use to analyze data and information, such as the
uses of machine-created information
• The processes humans use (for example, observing, reading, writing, taking notes, and
interacting) to obtain meaning from information
• Scientific theories of information content that allow its presentation in multiple forms and
formats without changing its value
• Usability, that is, ways to provide information that is easy to create, organize, access, and
use
• System use by multiple communities or user groups in multiple domains, in order to
establish general methods for building capabilities that transfer across communities and
domains
• Scientific theories underlying the use of models to generate data
• Developing and evaluating scientific, economic, and social domain models underlying information use 16
• The integration of cognitive principles of use in the specification, design, and
implementation of interactive presentations, such as visualizations, so that they are
appropriate for the information content
• Interoperability of data, information, and associated software
• Methods for designing, building, and maintaining systems for a usable, extendable IM
environment
• Methods for assessing the effectiveness of IM systems
b.
Managing Information as an Asset
On-line data and information provide both opportunities and challenges that require new
information management methods.
The greater complexity and richness of on-line materials provides opportunities. Of our five
senses, we most often use our eyes and ears, but are increasingly using touch, to interact with online
content. For each sense, content can be presented in multiple ways. Figures 1, 2, and 3
shown above of bits, code, and maps of ocean surface speed are illustrative. The HCI field refers
to sensory-based presentations as modalities and uses the word mode to denote different ways of
presentation within a modality.
A movie is multi-modal since it involves both sight and sound; text and images are different
modes of visual presentation. We need to develop ways to index, organize, and manage multimodal
content to be able to find it, provide it as requested, and allow user interaction with it.
Preservation provides a new challenge. Hardware platforms and software to read data on those
platforms rapidly become obsolete. While we can still read 500-year-old books today, how will
we access today’s on-line information ten years, let alone 500 years, from now?
Some information management issues lie outside the scope of HCI&IM R&D. One is intellectual
property rights, which may require legislative or regulatory action. Another is computing and
networking R&D, though those fields are strongly interdependent with HCI&IM.
HCI&IM R&D per se has its own rich set of topics for R&D in managing very large,
distributed,
heterogeneous multi-modal collections of data and information. These topics include:
• Digitizing legacy information and creating on-line descriptions of off-line material
• Cataloging, searching, finding, discovering, viewing, processing, and disseminating data and information
• Metadata and new ways to index and find information
• Multi-modal and multi-mode access
• Interoperability of data, information, and the software that accesses and uses them
• Guaranteeing 24/7 accessibility and dissemination
• Provenance, access and version control, accuracy and integrity
• Technologies for guaranteeing security, privacy, and confidentiality
• Long-term archival storage and preservation
• Designing, building, and maintaining information management systems
• Scalability of the collections and their management
c. Human-Computer Interaction and Interaction Devices
Humans use computing systems to augment their own capabilities. To do so, they interact with the data and information in those computing systems, in several ways:
• They seek content from information sources.
o The content can be presented in different modalities and different modes – visually (text, images), audibly (spoken and non-speech sounds such as music), or haptically (touch and pressure), etc.
• They interact with what they find.
o The means they currently use include writing or drawing, speaking or singing, pointing or touching, and moving hands or eyes.
o They use those means to further query the information, have it presented in different ways, or give directions (for example, stop a simulation, change some numbers, and restart the simulation using those new numbers).
• They control computing-enabled devices.
HCI&IM R&D investigates interaction capabilities to allow humans to use large bodies of
different types of information in better-controlled ways, and develops new means, such as new
devices, for interacting with computing systems. Today’s HCI&IM R&D focuses chiefly on
visual and audio combinations of presentation and interaction. Interacting with systems using
either written or spoken natural language processing has been a long-standing goal and HCI&IM
R&D area. For example, we want to be able to ask a computer a question, either spoken or
written, and have it answer that question. This will require substantial advances beyond today’s
technologies, in which we give key words to a search engine and usually get long lists of links in
return. All modalities and modes of interaction need further exploration to maximize a
computing system’s response to human action or to prompt a human to act. HCI&IM R&D
needs include:
• Basic understanding of the internal human perceptual, cognitive, and neural processes of
obtaining and using information
• Basic understanding and best employment of media, modalities, and modes to maximize
human ability to seek, access, and use information
• The science of usability, that is, of delivering information in a usable manner, which
includes providing optimal interaction capabilities for all possible human use
• Basic understanding of how humans share information and the methods they use
• Basic understanding of how groups of people work in a shared information space
environment
• Basic understanding of how teams (groups organized to work together) work in a shared
information space environment and how they evolved to become a team
• Basic understanding of how humans use information in individual and group or team
problem solving, planning, decision-making, and explaining
• Understanding how computing systems use data and information to maximize synergistic
human-machine capability
• Models of humans, computing systems, and the synergies between them to aid in
interactive system design
In their workshop presentations, mission agencies focused on the interaction needs of their
research communities. To that end, they are conducting R&D in interactive technologies
including:
• Decision support
• Designing interfaces for specific tasks and for multi-tasking
• Integrating user intentions into system or interface design
• Intelligent assistive devices and technologies ranging from handheld devices to robotic
assistants
• Interactions in multimodal and multimode environments
• Modeling presentation, use, and sharing of data and information
• Multimedia technologies
• Pervasive and immersive environments
• Security
• Spoken and written languages, including translation and speech to text
• Understanding of collaboration and development of collaboration technologies
• Universal accessibility
• Usability studies
d. Evaluation Methods and Metrics
The three HCI&IM areas discussed immediately above all require evaluation to determine
whether information technologies are successful and to suggest ways for overcoming any
deficiencies that are found. For both HCI and IM, there is a limited theoretical foundation. For
example, only a small body of research results exists on how humans interact with information,
such as what information humans use and how they use it to meet their needs. Rather, evaluation
methods applied to date have been adapted from scientific approaches in other disciplines.
Much of the evaluation methodology within the HCI community is based on experimentation
and the statistical analysis techniques employed by the psychological, social, and linguistic
sciences. Using these methods, measures are obtained to quantify total (human, computing
system, and their interaction) system performance such as time to process or quality of results.
One overarching goal of this work is to develop models of cognitive processes that, in
combination with models of total systems, can predict the properties of an operational design
before any
implementation begins. Today, a design can be evaluated directly only in the deployed
experimental state.
IM evaluation methods have been taken from the information retrieval community. These
include methods for assessing the relevance of the information returned in response to queries
and methods for assessing technologies that summarize the content of retrieved material. A
metric used for the latter is similarity between the content of summaries generated by computing
systems and those generated by humans. Such metrics have been used in the DARPA/NIST Text
Retrieval Conference (TREC) evaluations. Similarly, scientifically derived assessment tools are
needed for other information management areas.
Methods for using information in today’s computing systems include artificial intelligence
approaches to formal logical, Bayesian or other statistical methods of inference, and neural
networks and genetic algorithms for classification and categorization. One defect of logical
approaches is that while they can produce logically valid inferences, the “information” within
those inferences may be false. Before incorporating information produced by a computing
system into an on-line data or information source, evaluation criteria must be applied to
determine whether or not that information is correct. In the physical sciences such as biology,
chemistry, and physics, concrete measures are used to validate information. In abstract worlds
outside the physical realm, human uses determine if information is valid, and in this abstract
world of information, evaluation challenges abound, as seen in the following research needs:
• Theory of evaluation for IM
• Theory-based evaluation methods for HCI&IM
• Theory of evaluation for interactive information systems
• Theory of information validation
• Metrics of total system performance
o Under basic and a variety of other conditions
o Error bounding criteria for system acceptability and usability
• Predictive methods for human performance while using computing systems
• Models of total system function
Methods need to be developed to validate:
• Models of natural systems:
o Physical systems such as weather
o Biological systems such as DNA and cells
o Cognitive systems and performance
o Groups and teams
• Mathematical models of analytical methods
• Models of properties of information content such as uncertainty and error propagation
during inference
• Models for evaluating design before implementation, especially where the human is a
critical link in system use
Sumber : https://www.nitrd.gov/pubs/hci-im_research_needs_final.pdf
