Updated 3/14/03
USER STUDIES AND EVALUATION
Jim Blackaby
Director, Internet
Strategies and Information Services, Mystic Seaport
While we reach lots and lots
of people with digital content, and we can capture a great deal of information
about their browsers, the link that brought them to us, how long they seem to
pay attention to things, and so on, we don’t know a lot about how to make good
sense of that data or to really be able to have an idea of what our users are
thinking about or how well we are accommodating their needs (or even what their
needs are). Usability studies at the
outset are valuable just to make sure that our audiences understand the
interface, its options, and so on, but I am not aware of a great deal of work
that has been done beyond this basic level.
Allison Druin
Digital Tools and Resources for the People, by the
People
We need to ask two critical
questions when considering the creation and impact of new digital tools and
resources: (1) how can technology have an impact on the lives of users and (2)
how can users have an impact on the development of new technology? With each we need to consider both the
research methods we use to answer these questions and the best ways to
communicate these results to make change for the future.
In particular, we see two
questions in this area that should be addressed:
(1) How do
people change in how they see books, libraries, technology and other cultures
thanks to new digital tools and resources?
It’s no longer enough to
ask—how fast can someone access a digital book? Or how many times does a person
use digital materials? We should
instead be exploring the causes for the speed or use. We need to understand if these digital tools make a difference in
how people live their lives and see their world. We need to consider what matters to people and how they perceive
their knowledge tools.
(2) What
evaluation methods can we use to deeply explore these issues?
We need evaluation methods
that help us tell the whole story.
Researchers should not be afraid to use many different approaches to
understand how digital tools and resources have an impact on people’s
lives. From surveys, to participant
observation, to online log files, all can be critical in helping to explore
this impact.
In this area, we again see
two further questions for research:
(1) How can we develop information resources
and tools that support users in their particular context for their specific
needs?
Not all users were created
alike. For example, when people suggest
they have created a website for K-12 use, it disregards the differing needs,
abilities, and experiences children have at different ages. Instead, we need to understand that our
technology interfaces must be configurable and appropriate for diverse people,
with varying cognitive abilities, technology experiences, and
social/work/learning needs.
(2) What
are the roles that users can play in the technology design process?
There are numerous methods
and times that users can be involved in the design of new technologies. We can hear from users at the start of the
idea generation process, and throughout the prototype development. Researchers need to understand how they can
create opportunities to hear the needs of users in order to create more
appropriate technologies for the future.
Ross J. Loomis
Colorado State University
Suggested areas of research
related to users of websites include the following:
References
Eastin, M. S., & LaRose, R. (2000). Internet self-efficacy
the psychology of the digital divide. Journal
of Computer-Mediated Communication. 6. (1).
Fonda, S. (2002). Exhibit
planning resource workbook for Collage Children’s Museum, Boulder, Colorado.
Unpublished project paper, Boulder, CO: Museum and Field Studies, University of
Colorado.
Johnson, N.B. (2000). Tracking the virtual visitor: A
report from the National Gallery of Art. Museum
News, 79(2): 42-71.
Loomis, R. J., Elias, S. M., & Wells, M. (2003).
Website availability and visitor motivation: An evaluation study for the
Colorado Digitization Project. Unpublished Report. Fort Collins, CO: Colorado
State University.
William E. Moen
Associate Professor, School
of Library and Information Sciences, Texas Center for Digital Knowledge,
University of North Texas.
State libraries are
expanding their traditional service offerings by building statewide virtual
libraries with resource discovery services that take advantage of the
intersection of portal technology and user demand for networked resource
discovery services. These services are enabled by standards-based access from
users' homes and offices to a wide range of digital (and analog) resources in
libraries and repositories throughout a state. Other libraries and library
consortia are implementing similar resource discovery services to integrate
access to diverse resources. We can characterize these services as providing a
single search interface (but possibly multiple user interfaces to the search
interface) to a diversity of databases and metadata repositories (e.g., online
catalogs, commercial online databases, OAI-created metadata repositories,
special collections described in various metadata schemes, etc.). Sometimes the resource discovery service is
called “broadcast search” or “distributed search.”
Much as we might transfer many of the services of a
physical library environment to a distributed searching environment, the
interface(s) to access virtual library services may be quite different from
physical library access of the past. Our initial conception both of the
services and the interfaces to the virtual library are based in large part on
our collective experience in the traditional library. We can expect that as
access to the virtual library enters the homes of citizens, current and
potential users will define new requirements for the resource discovery service
interface. Now is an ideal time in the maturation cycle of the statewide
virtual library concept to identify the characteristics of effective interfaces
for virtual libraries offering diverse distributed resource collections to
their citizenry. Understanding the information behaviors of various user groups
and optimizing resource discovery interfaces are critical to the success of
statewide virtual libraries.
The approach we are taking
at the Texas Center for Digital Library is to develop methodologies for
usability assessment of the interfaces that enable user access to multiple and
diverse resources. Focus groups conducted by the Center staff in conjunction
with the Z Texas Implementation Component of the Library of Texas (ZLOT)
Project in the spring of 2002 indicated that different user communities exist
within the state and that (a) the needs of these communities differ radically
and (b) an individual could belong to more than one group. Additionally, we can
anticipate that both existing library patrons and new library patrons will
interact with the statewide virtual library. This expectation predicates the
following questions:
·
Who are the users of
statewide virtual libraries?
·
In addition to existing
library patrons, what new market segments can be reached?
·
How do the resource
discovery patterns and information needs of various user groups differ?
·
Are there user group
differences based on information literacy variables, demographic variables, or
technology adoption variables?
·
What design characteristics
will optimize utilization of statewide virtual libraries across a wide range of
citizenry?
Diversity in users’ information seeking behaviors
and needs requires application developers to design interfaces for virtual
state libraries that are usable, where usability criteria may include
efficiency, effectiveness, engagement, error tolerance, ease of learning (from
Quesenbery’s five E’s of usability).
Optimization of a user interface to a service
requires knowledge of virtual library user groups and any differential
requirements among the groups. Since statewide virtual libraries are emerging
phenomena, identification of the usability requirements of their user groups
has been largely speculative in nature and often based on knowledge of existing
library patrons. This is an ideal time to engage in research to identify the
characteristics of effective user interfaces to statewide virtual libraries.
This type of research could not have been conducted at an earlier time.
Debora
Shaw
Indiana University
A persistent trend in user studies research over
the past 20 years has been our increasing appreciation for the complexity of
understanding "users." We have brought in methods and perspectives
from psychology, sociology, anthropology, etc. We know that demographics,
previous experience, and immediate context, among other factors, influence user
behavior; but we don't seem to have a good sense of how all these
interact. Nor do we understand how (or how much)
any of these 1) affect information services in libraries, or 2) affect
information seeking and use online/in digital libraries.
Attempting to understand the cognitive,
psychological, social, and cultural aspects of information seeking and use is a
worthy challenge, and probably a guarantee that many researchers will be
gainfully employed for a long time. In the meanwhile, people go on seeking
information all around us, and librarians and others function remarkably
effectively as intermediaries. Researchers would do well to investigate the
tacit knowledge held by individuals and exchanged in communities of practice.
What I have in mind is the kind of detailed,
ethnographic work Nardi and O'Day report in their chapter on libraries in
Information Ecologies. Such an approach would emphasize researchers studying
and understanding the world of practice; this would be different from previous,
somewhat artificial attempts to have practitioners and researchers collaborate
on an external project of mutual interest.
Possible useful research questions are very basic;
they include: How do the physical
structure of a library and the arrangement of collections (and other resources)
affect information services? What really happens in question negotiation (from
the librarian's and the client's perspectives)? How do reference librarians determine which sources to consult in
answering a question? How do the participants know when a reference interview
is over?
INTEROPERABILITY
Jim Blackaby
Director, Internet
Strategies and Information Services, Mystic Seaport
In the February posting of
the NSF I-2 News, there was a discussion of “a future cyberinfrastructure that
will ‘ radically empower’ the science and engineering community.” The interesting thing about the discussion
was that in the museum and library community we have spent years developing
tools for interoperability – some of which (those in the library) have been
successful in serving particular vertical components and some of which (those
in the museum) have been successful in modeling complex information across a
range of disciplines, even if those models have not been well realized. In spite of that, we are hardly on the edge
of being “radically empower[ed]” or having a sense of our
“cyberinfrascturcture.”
Tim Cole
University of Illinois at UC
The range and scope of
potentially valuable research that’s waiting to be done in the general area of
digital library interoperability is vast. For me as a practitioner the most
important unresolved interoperability issues are in two key areas: metadata use
and content identification and linking.
In regard to metadata use we
need additional, more generalized studies of how cultural heritage institutions
create and use metadata and in particular methods by which such organizations
can better create metadata that at once meets the needs of local
implementations and communities and the requirements for many kinds of robust,
high-level interoperability. This requires research that models the metadata
creation process, perceptions about metadata, and the decision making that goes
into selecting and implementing metadata schemas. Work is needed to better
enumerate the ways (and differences in the ways) metadata is used in both a
single institution local context and a multi-institution collaborative
environment. What is the generalized definition of ‘good’ metadata in
interoperability context? (We know it when we see it, but can we articulate a
useful definition that fits across the communities and sub-communities of
libraries, archives, and museums?) What are the real costs of generating
quality metadata, especially as a function of its utility for sharing? How can
the bar (and cost) be lowered to better enable smaller institutions generate
and share metadata? Some solid community-specific early work has been done in
these areas (e.g., Guinchard, 2002), but more is needed. The potential payoff
is high. By generalizing what is meant by good metadata, we can move away from
project-specific solutions, where an institution invests in one project’s
interoperability metadata solution only to find it incompatible with another
project’s approach to metadata interoperability. Create the metadata once,
reuse many times.
Also related to metadata and
its usefulness in context of interoperability, we still need additional
research into the utility of aggregated metadata. Methodologies for studying
usability and usefulness of search and discovery tools is well understood, but
the figures of merit used for single project, homogenous implementations aren’t
necessarily as meaningful in a collaborative, heterogeneous implementation. If
user satisfaction with a portal that searches metadata aggregated from 100
repositories is high as measured using current methods, but all retrieved
records come from only 1 or 2 of the repositories represented in the
aggregation, is that a ‘good’ interoperability service? Do such results prove
the value or benefit of metadata sharing? Arguably not. So what is an
appropriate measure of useful metadata sharing? Are there new measures that
have to do with how well metadata aggregators use metadata to discover and
reveal relationships between and among collections and items?
As the richness of Web
content continues to grow, de-duplication and precise identification of
resources will be increasingly important priorities. Already it’s common in
Google searches to see on a single screen of results multiple links to the same
work in different formats (e.g., HTML and PDF) and/or at different (often
aliased) URLs. The converse also occurs, such that a resource in one format is
found, but the same resource, in a more convenient or complete format, is not
found. This suggests that there needs to be new research that builds on,
extends, and brings together research into the appropriate copy problem (e.g.,
Beit-Arie et al., 2001) and research on functional requirements for
bibliographic records (e.g., Hicky, O’Neill, and Toves, 2002). Both existing
research threads recognize the potential for duplication of content. Research
is needed to further extend these models from the traditional bibliographic
realm into the realm of online primary content. There’s also a related need for
research that extends existing models for quorum or best-matching searching of
bibliographic records into the realm of online full-text/full-content and for
research that postulates and then demonstrates new kinds of services that can
be built to better link together related resources to the benefit of
information users.
Guinchard,
C. (2002), “Dublin Core use in libraries: a survey”, OCLC systems & services. v.18, no.1, pp 40-50.
Hickey,
Thomas B., Edward T. O’Neill, and Jenny Toves (2002). “Experiments with the
IFLA Functional Requirements for Bibliographic Records (FRBR)”, D-Lib Magazine 8, no. 9, online. http://www.dlib.org/dlib/september02/hickey/09hickey.html
Beit-Arie,
Oren et al. (2001). “Linking to the Appropriate Copy: Report of a DOI-Based
Prototype”, D-Lib Magazine 7, no. 9,
online. http://www.dlib.org/dlib/september01/caplan/09caplan.html
Anne Craig
Associate Director, Library
Automation and Technology
Illinois State Library
The Illinois State Library,
in keeping with its mission to meet the information needs of Illinois citizens,
remains continually focused on public access to state government
information. While procedures to ensure
that access to printed publications have been developed over decades of
practice, the parallel methodologies and tools are yet to be developed for
electronic access. Two areas of
successful IMLS grant projects constitute a successful beginning. First, the state GILS projects, which were
created by a National Leadership Grant to the Washington State Library, provide
multiple access points for precision searching of state Web sites. For an
idea of the status of this work in different states, see http://gils.utah.gov/otherstates.htm. Second, Preserving Electronic
Publications (PEP), a National Leadership Grant to the Illinois State Library,
created a replicable system for capturing and caching state online documents by
developing a software system tailored to state government needs and making it
available for free download. The PEP information
is available from http://www.isrl.uiuc.edu/pep/,
and the official PEP Web site is http://www.cyberdriveillinois.com/library/isl/lat/pep/pep.html. Significant work remains to be done to build on this solid foundation.
As each state develops access to its own electronic
documents, awareness is arising of the need for interstate interoperability to
pool information resources. Experience
as reference librarians taught us that interstate searching held real value for
our patrons. In state government, the
first reaction to a need is often, "How are other states handling
this?" State GILS participating
institutions have begun to realize what a service could be rendered to their
citizens if precision retrieval could be applied across state lines. The FirstGov Web site demonstrates
recognition of the value in grouping state information for the user.
A study of possible avenues for interoperability
among state GILS programs would move the states toward realizing the ideal of
broadcast searching from any state's GILS site to any or all of the others that
is seamless from the user's point of view.
Metadata provides the foundation for interstate interoperability. Therefore, the metadata would likely need to
be enriched, possibly in a separate, associated file, which would require
hiring metadata creators. Subsequent
trials would determine whether state GILS metadata is sufficient to support
interoperability, or if a crosswalk and the Open Archives Initiative protocols
would be desirable for best cross-database access.
Bernie Hurley
Director of Library
Technologies, UC Berkeley Library
From a macro-view, one could
argue that digital libraries need to perform three functions: content
management; preservation and end-user access.
Content management services allow for the creation, ingestion and
maintenance of digital objects.
Preservation services provide for the long-term retention of objects,
while access services are used by the end-users to discover, display and
manipulate materials from the digital library.
A key question may be how
tightly these three services should be bound together in a digital library
architecture. Is it possible to try to
provide all three services from a single system? Or, will there be many different content management systems
(CMS), perhaps each one specializing in a different format (e.g., text, audio,
video, datasets, etc.)? Can we build
access systems (AS) that can “do everything for everybody?” Or, are access systems transient services
that are built around a preservation repository by communities that have the
need and the funding, but are abandoned when the need or funding evaporates?
If one assumes that we are
going to build megalithic digital library systems, interesting interoperability
questions arise. Particularly, what are
the relationships between these systems and how do they interoperate?
1) Where do the
digital objects reside? This is a
basic question, but one that becomes interesting as digital objects grow in
size (e.g., audio, video, satellite datasets, etc.). We probably won’t want to be storing these multiple times in
content management, preservation and access systems. Also, having multiple copies confuses the issue of which one is
the “master” object.
2) How are
access systems built if they don’t have copies of the digital objects? Are access systems created by mining metadata (for
discovery services) from content management or preservation systems? How do the access systems interact with the
other services to “get” an object to be displayed to a user?
3) At what
granularity does data need to be moved between these systems? Clearly we don’t want to be moving large digital
objects across the network when there is only a need to use part of the
object. For example, a 30 second
segment from a large video file.
Assuming one agrees with the
above, the technical research questions become. What kinds of protocols and technology standards do we need to:
1)
Allow repositories to “discover” each other and set up connections for
exchanging digital materials (including the mining of metadata for building
access systems)?
2)
Move an entire object, or pieces of a digital object between systems in
real-time?
3)
Translate materials on the fly between digital object encodings used by
different communities (e.g., METS, IMS)?
William E. Moen
Associate Professor, School of Library and
Information Sciences, Texas Center for Digital Knowledge, University of North
Texas
Optimization of Interoperability in a Broadcast
Search Environment
As noted by Lynch and
Garcia-Molina in their 1995 report from the Interoperability, Scaling, and the
Digital Libraries Research Agenda workshop, interoperability is multi-faceted,
and achieving optimal interoperability is a key challenge in the networked
environment. The broadcast search environment enabled through virtual libraries
and portals increases the complexity of interoperability as users attempt to
engage with a variety of separate, distributed information retrieval systems,
multiple metadata schemes, various protocols, controlled vocabularies, etc.
Tools and technologies, such as XML, Dublin Core, Z39.50, provide a foundation
for interoperability – protocol, syntactic, and even semantic interoperability.
In the broadcast search environment, the diversity
of the installed base presents difficulties in achieving meaningful
interoperability. This installed base consists of separate, diverse information
retrieval systems and their associated databases. Ongoing research in the
Z39.50 Interoperability Testbed Project, funded by IMLS, at the Texas Center
for Digital Knowledge is demonstrating that some types of interoperability can
be improved (protocol, syntactic, and database semantics) through community
agreements related to how a specific protocol will be implemented, which syntaxes
for exchanging records will be supported, and indexing policies of relatively
homogeneous databases (e.g., MARC-based bibliographic databases). This research
is also highlighting the effects on search and retrieval caused by differences
in information retrieval systems, such as data normalization rules,
word-extraction processes, and character set encoding and normalization in
records and indexing. Most recently, we have seen variation in indexing
algorithms for phrase-oriented searching and the handling of leading articles.
These differences affect search and retrieval interoperability. To date, the
research shows that improvement in interoperability for basic keyword searching
is achievable, but advanced searching remains problematic. And this is in an environment
of relatively homogeneous information retrieval systems, databases, and record
content (namely, online catalog systems and associated MARC bibliographic
databases).
The questions surrounding interoperability
invariably involve the development and use of appropriate community agreements
(e.g., technical standards, guidelines, and practices). We have suggested
before that the costs to achieve interoperability are likely to be less within
an information community (e.g., libraries) because of agreements within a
community. Yet, the research mentioned above indicates that even within such a
community, optimizing interoperability remains a challenge. Improving
interoperability through community agreements allows us to deploy first
generation broadcast search services, but the next step is to understand users'
expectations for interoperability. Several questions come to mind:
·
Is there a way to discuss and measure optimal
interoperability from a user's perspective?
·
When is interoperability "good enough"
from a user's perspective?
·
How will different user groups' information seeking
behavior and needs be reflected in different criteria for optimal
interoperability?
Resolving the threats to interoperability in the
networked environment appears to require work at both the technical and the
social/organization levels. The tools, technologies, and technical standards
must be addressed. Even with a technical standard (e.g., Z39.50, XML, Dublin
Core), the implementation of the standard may differ between providers.
Organizations' local decisions when implementing these tools and technologies
also can have adverse effects on interoperability. In the networked
environment, a new balance has to be struck between serving local users and
others wanting to engage with the local resources. Or, what is a beauty mark
for a local user of a system turns into a wart for others! The providers of the
tools and technologies want to differentiate their products in the marketplace;
however, if their products exhibit characteristics that reduce
interoperability, the broader community may find the less-than-optimal
interoperability unacceptable.
The questions of optimizing interoperability point
to the need for more community agreements on deploying technologies, tools, and
resources. The challenge is to identify the community agreements needed and put
in place methods and procedures for developing and implementing them.
Tom Moritz
American Museum of Natural History
Markets/Law and Norms: a
Problem Statement
In the market/economic
realm several interrelated problems seem clear. First, on the supply side,
though information may want to be
free, it has inevitable costs (a colleague who works for a major international
information-based non-profit has noted: “No margin, no mission”). (This means, particularly in difficult
economic times, that organizations must optimize opportunities for revenue and
my be criticized as maladaptive if they fail to “leverage” every available
asset. And more simply, many
organizations simply can not undertake digital projects because they lack
“venture capital”. ) Broadly speaking
this suite of issues falls under the rubric of “sustainability”.
On the demand side, while
the market is exceptional in its ability to mediate human self-interest (see Adam
Smith), it is often unfair
particularly in a pluralistic society that seeks fairness as one of its most fundamental principles (see John
Rawls). But fairness is not the only
value impelling open access. The
authors of the U.S. Constitution recognized such potential value in information
and knowledge that they contravened well established property law by requiring
the reversion of intellectual property (information) to the public domain -- the public good that flowed from this was
seen to be the nurture of innovation and creativity. In the United States, both libraries and museums take service to
the public good as primary mission. In
fact it is very plausible to argue that in the United States, both libraries
(at least public or quasi-public libraries) and museums are creatures of the public domain.
Beyond all these
considerations, on the global scale, and in the United States as well, there
are stark inequities of wealth that ramify in both an information gradient (analog and digital) and in a digital divide (technological). These are fundamental issues of fair and
equitable access to information.
(These problems are in turn
amplified by law and policy as well as by social and cultural norms particularly as these norms relate
to perceptions of property rights and contradicted by codes of ethical
conduct.)
Two possible lines of
research arise from this problem statement:
First, with respect to sustainability, what models are available to sustain the financial viability of
digital developments? Which models have
proven successful? Which have failed?
How can the extensive experience of American business practice and the
expertise of American business education, be applied to these researching these
problems?
Second, what models exist
for removing economic/market, legal and normative barriers to
interoperability? A number of
innovative approaches are already In various stages of implementation. Among these are: The U.S. National Science Digital Library, The Creative Commons,
The Biodiversity Commons, The Public Library of Science, the World Health
Organization’s HINARI Project, BioMed Central, BioOne,
FreeMedicalJOurnals.com. Are these
models scalable and/or extensible? Can they be rigorously analyzed in terms of
long term sustainability?
David Seaman
Director, Digital Library
Federation
The scope of the issue: The
"data silos" problem is a misery that keeps on giving. We need to exploit all available standards
and technologies to help us infuse malleability, nimbleness, interoperability,
and repurposing into our use of (and assumptions about) the digital library
object.
Provide 3-5 priority
problems (research topics), indicating why each is important and what solving
it will accomplish.
* Innovative users need
innovative, nimble content that fosters discovery, engagement, and experiment.
Our users are too often invited to watch "content channels" whose
aesthetics, services, and behaviors are dictated simply by the terms of the
creating institution or publisher, rather than being able to download,
manipulate, morph, annotate, cross-search, and repurpose digital library
content (the "music mix" model).
* Integration of remotely
held content into local library services, courseware systems, and data analysis
tools is repeatedly thwarted by the behaviors of the digital repositories. We
too often face content that does not support the creation of end user
services. At the extreme, we can't even
link at the article/chapter level to some items, forcing libraries to buy
access to digital content, then also to re-digitize locally from the paper copy
to incorporate into an e-reserve service or online syllabus.
* Our digital library
objects are underachieving -- behaving way below their potential as malleable
items with which the user can engage actively.
This poor data behavior contributes to users who are under-ambitious in
their use of the digital library.
* Libraries who create
content are part of the problem as well as a part of the solution. We too often create silos of our own locally
digitized material, and we are not articulating the problem clearly enough to
the publishers and vendors we buy from.
They think they are giving us what we want.
APPLICATIONS/EMERGING TECHNOLOGIES
Jim Blackaby
Director, Internet
Strategies and Information Services
Mystic Seaport
Museums and Libraries have
more stake than many other information managers in dealing with large digital
objects – whether high resolution (as in the case of maps or works of art),
extent (as in the case of digital audio and video), or both (as in the case of
architectural drawings or manuscripts).
In spite of that stake, we have lacked presence in the research sector
to have our needs addressed particularly well or consistently. While it is true that we cannot offer the
commercial opportunities that others might, once some tools were developed to
enable using these kinds of materials, those would be useful for all.
Anne Craig
Associate Director, Library
Automation and Technology
Illinois State Library
The Illinois State Library,
in keeping with its mission to meet the information needs of Illinois citizens,
remains continually focused on public access to state government
information. While procedures to ensure
that access to printed publications have been developed over decades of practice,
the parallel methodologies and tools are yet to be developed for electronic
access. Two areas of successful IMLS
grant projects constitute a successful beginning. First, the state GILS projects, which were created by a National
Leadership Grant to the Washington State Library, provide multiple access
points for precision searching of state Web sites. For an idea of the status of
this work in different states, see http://gils.utah.gov/otherstates.htm. Second, Preserving Electronic Publications
(PEP), a National Leadership Grant to the Illinois State Library, created a
replicable system for capturing and caching state online documents by
developing a software system tailored to state government needs and making it
available for free download. The PEP information
is available from http://www.isrl.uiuc.edu/pep/,
and the official PEP Web site is http://www.cyberdriveillinois.com/library/isl/lat/pep/pep.html. Significant work remains to be done to build on this solid foundation.
State and local governments need search engines to
facilitate citizen access to electronically published information. Although commercial search engines are
available, there are at least two reasons that the private sector is not
meeting this need. First, the
overwhelming majority of state and local governments do not have the financial
resources to purchase and maintain software packages that cost $500,000 in
first year fees and require correspondingly high annual maintenance and support
charges. Second, government represents
too small a market for private sector organizations to expend research and
development resources in order to tailor a product to public sector needs. Therefore, a NLG that would develop and
distribute an open source search facility software would be of great lasting
value to government and citizenry throughout the country. The needed software would be capable of operating
on parallel architecture such that a system would be expansible by adding
multiple computers without need to re-balance the workload.
NON-TEXTUAL FORMATS
Jim Blackaby
Director, Internet
Strategies and Information Services
Mystic Seaport
There are two sides of the
non-textual issue – storage and access on the one hand and delivery and
presentation on the other – that want consideration. For museums both issues are clouded by the fact that almost
nothing is entirely non-textual. A
Stanley 45 Combination Plane in a museum collection when digitally captured is
not without a bunch of text that might help group it with other things. Image recognition algorithms for digital
photographs may be helpful for some types of research, but for museum collections,
any knowledge about the image is apt
to trump the results of image recognition.
Several things might be interesting to explore:
Virginia G. Fox
I’m
so delighted that IMLS is providing leadership in public media digital issues. Those of us in public television,
particularly those that serve both the general public and very particular K-16
interests, are really in need of model generation, research and guidelines. We have significant cataloging, preservation
and rights issues that we feel would be better resolved in collaboration with
other public media institutions that share our mission than the world of
commercial film and television.
Cataloging
issues:
What
are the standards for cataloging digital video, particularly television? What granularity is recommended for
effective use and cost effectiveness?
Teachers and students seek information on particular skills or concepts
within a broad subject area. There may
be several of these concept pieces within any given program. Public television is in a good position to
lead this process because we caption all of our programs. However, key word specification is the tip
of the iceberg in describing/cataloging/creating metadata for the pieces. Money is needed to describe things at a more
specific level than we are used to, but we also need to know much more about
what users are looking for and how they search.
Preservation
issues:
How
long do DVD’s, hard drives, servers, digital tapes last? We need the cooperation of the manufacturers
in making these media more stable. How
often should electronic information be transferred to new media? Do we have model cost studies? As more and more contemporary media is
created digitally, reformatted digitally and stored digitally, we need a new
working definition of preservation.
IMLS can lead this redefinition.
Rights
Issues:
Copyright
rules governing sound recordings and moving images and rights expirations are,
of course paramount. We desperately
need model rights agreements. Archives
and museums have the same issues with donated materials. Often the donor family has no idea of the
rights to their own collection. Record
keeping in these matters on the part of all public media institutions has been
hampered both by lack of funding but has been exacerbated by the emergence of
access media unimaginable at the time much of their collection was
created.
All
public media need help in how to determine what the rights are “after the fact”
and come to consensus on best practices in providing access to the “we don’t
own the rights and don’t know who (if anyone) does” materials.
PRESERVATION
Jim Blackaby
Director, Internet
Strategies and Information Services, Mystic Seaport
Whether the issue is
planning for digitization, digitizing, maintaining digital materials, or
distributing digital materials in rich contexts, digitization requires
significant resources – collections, staffing, expertise, time, and money. The upside, of course, is that once an
effective strategy is in place, it is relatively easy to maintain and
re-use. The challenges of doing the
digitizing encourage collaboration. The
benefits of meeting those challenges address the notion of preservation –
perhaps not in the sense of keeping things from disintegrating over time, but
rather in the sense of preserving their utility.
COLLABORATION
Jim Blackaby
Director, Internet
Strategies and Information Services, Mystic Seaport
Whether the issue is
planning for digitization, digitizing, maintaining digital materials, or
distributing digital materials in rich contexts, digitization requires
significant resources – collections, staffing, expertise, time, and money. The upside, of course, is that once an
effective strategy is in place, it is relatively easy to maintain and
re-use. The challenges of doing the
digitizing encourage collaboration. The
benefits of meeting those challenges address the notion of preservation –
perhaps not in the sense of keeping things from disintegrating over time, but
rather in the sense of preserving their utility.
Virginia G. Fox
Collaboration
among all public media entities on the digital issues outlined in the
invitation to this workshop is critical to assuring access by the public to our
individual and collective assets.
I believe that the
advantages of collaboration will be maximized if all participants will specify
the terms and conditions for agreeing to the following statements.
There
are two very general, but timely and strategic advantages to collaboration.
The
most compelling collaborative issues are:
The
advantages to collaboration in these areas include:
Bernard F. Reilly, Jr.
Center for Research
Libraries
Ensuring the long-term
accessibility, or persistence, of digital resources poses substantial
technological challenges, requiring reconciling the inherently dynamic
characteristics of digital objects (and the technologies that support them)
with the permanence that is the goal of preservation. The scale of investment required and the nature of digital
resources will require that these challenges be addressed collaboratively.
Partnerships, alliances, and consortia provide some of the supporting
frameworks through which universities, libraries, museums, publishers,
government agencies, and other organizations work together to produce and
maintain digital resources. Within
these frameworks investments are made, risks and responsibilities apportioned,
and returns managed. These
collaborations vary considerably in structure and governance, and in the kinds
of economic models and practices that they adopt. Some structures and practices are better suited than others to
minimizing risk and ensuring the appropriate stewardship of resources.
It might be worth examining
this side of resource-building partnerships in connection with digital
content. The Center for Research
Libraries and some of its partners have been looking at a number of models for
cooperative resource-building, in connection with a survey of how library
collection development efforts control risk, and an investigation on how to
archive Web-based political communications.
(The latter is funded by the Andrew W. Mellon Foundation.). We are beginning to better understand how
important the organizational models and methods of financing are to the
resources they support. We would
welcome a chance to explore this further and learn from the experience of
others at the workshop.
Debora Shaw
Indiana University
The buzz-initialism for
collaboration is CSCW: computer-supported cooperative work. Each of these words
suggests potential research questions: Computer: How to people collaborate
without computers? Research on collaboratories, for example, often takes the
connections via information and communication technologies (ICTs) as a given;
how do people work together when they are co-present?
Supported: What about
failures of collaboration what does the
computer disrupt or diminish? Might there be instances of computer-sabotaged
cooperative work that would, by contrast, improve our understanding of what
works well?
Cooperative: Not all
collaborations are cooperative. How can ICTs be used to manipulate co-workers,
hijack a group's deliberations, damage individuals or the group? And what can
be done to minimize such negative activities? Study of groups dynamics is
clearly essential, but is/how is increasing computerization (information
overload, global competition, system complexity) affecting how groups work?
Work: We have learned some
things about using computers for recreation or education; these insights and
connections may be increasingly important as new generations of workers and
post-9/11 sensibilities affect how people think about their priorities and
expectations of both work and play.
Readings
Nardi, B. A., & O'Day,
V. L. (1999). Information ecologies: Using technology with heart. Cambridge,
MA: MIT Press. See especially chapter 7, Librarians: A keystone species (pp.
79-104).
GENERAL
Diana Folsom
Manager,
Art & Education Systems, LACMA
My thoughts and concerns go
across the topics broadly, rather than one topic in depth. Here are some
notes:
Our museum is probably
typical of most museums, in that we are still at the basic level of cleaning up
core records and getting as many records and images online as possible. Our goal has been to have a record for every
object available online, along with an image for as many as copyright will
allow. We currently have 45,803 records
and 31,719 images online in a search tool that we are working hard to make
useful for this large quantity, and many audience levels. We have a collection of 110,000 objects, so
we still have lots more basic work to do!
Funding and staffing is
limited, so we must make the best use of our time and resources. Although we
are willing to experiment a little, we need good peer advice and user
evaluations to guide our work so that we can work smart. We've been getting small amounts of
information from the IMLS user study
(Minneapolis Institute of the Arts) and it is extremely useful. As soon as this study has been compiled, it
should be distributed widely. We will
all help get the information out!
We haven't tackled
interoperability issues yet and would like to feel relatively confident that
whatever metadata and new coding we do (XML?) - that it will work for many uses
and for a long time in the future.
It seems that many different
special interest groups are trying to create portals. There is an attempt to cluster information in various arenas, but
wouldn't it be wise to have some kind of master plan within the arts and
humanities?... I scan write-ups about
various plans, projects and ideas but I rely on advice from colleagues to help
with my assessments. Projects are
fairly well publicized in the beginning, but results are harder to find. It is
difficult to keep up with new research while we are working in the trenches.
We
are lucky to get cooperation between departments within our museum, let alone
venturing outside the walls to work with anyone else. It feels like there is a huge chasm between getting our core data
online and working with another organization.
We would like to know about
good software tools that are easy to use. ..."middleware" of various
types and how they can best be used ...harvesters, methods for mapping and
clustering terms, metadata schemes...
We would like to think about
basic issues in new ways. We are most
interested in integrating information about artwork with information from
libraries in meaningful ways. In other
words, might we juxtapose images and writing from the same period with or
without additional interpretive writing?
In addition to curation, are there intelligent, semantic tools that
could help?
John Perkins
CIMI
Rather
than tell you what I personally think are the major research issues in your
topic groupings I have three consultative efforts to share with you.
1.CCF report on a Survey of
International Research in Digital Cultural Content
CIMI, Resource and JISC sponsor an international group called the Cultural Content Forum http://www.culturalcontentforum.org/. Our specific focus is on trying to identify common pan-national issues and solutions in the general area of digital cultural content. At our meeting in Washington a year ago (attended by IMLS and a number of your s