Moving Image Collection Evaluation

Online instruments

Chen, P. User satisfaction of using Stanford's library systems: Socrates. Retrieved June 16, 2003, from: http://hci.stanford.edu/cs377/schiano/SampleSurveyPilot.pdf.

DIRECT questionnaire. Retrieved April 6, 2003, from http://www.library.usyd.edu.au/borrowing/docdel/questionnaire1.html

Formative Evaluation Questionnaire of HealthCyberMap Pilot Implementation. Retrieved April 6, 2003, from http://healthcybermap.semanticweb.org/questionnaire.asp.

France, R. B., Nowell, L. T., Fox, E. A., Saad, R. A., & Zhao, J. (2003). Use and usability in a digital library search system. Retrieved April 6, 2003, from http://www.dlib.vt.edu/Papers/Use_usability.PDF.

Fuhr, N., Hansen, P., Mabe, M., Micsik, A., & Solvberg, I. (2001). Digital libraries: A generic classification and evaluation scheme. Retrieved June 1, 2003, from http://citeseer.nj.nec.com/fuhr01digital.html.

Hartland-Fox, B., & Dalton, P. (2003). eVALUEd - an evaluation model for e-library developments. Retrieved April 6, 2003, from http://www.ariadne.ac.uk/issue31/evalued/.

"We are particularly concerned with the issue of evaluation, e.g.:

• what techniques are being employed
• who uses the data collected
• how evaluation can inform decisions
• what evaluation could be conducted given more time, resources, staffing etc."

The questionnaire is in http://www.cie.uce.ac.uk/evalued/Questionnaires/eval_question.htm

K-12 Maryland Digital Library questionnaire. Retrieved April 6, 2003, from http://www.bcpl.net/~dcurtis/digital/quest.html

List, C. (2003). Questionnaire design for the Evaluation of the Museum Education Site Licensing Project. Retrieved April 6, 2003, from http://sunsite.berkeley.edu/Imaging/Databases/Fall95papers/list2.html

"The next step is then to examine each of these potential reports to establish what information the MESL organizers hope to gain. To take the first example, a list can be made of the possibilities for findings.

• Who are the instructors?
• What departments are the instructors affiliated with?
• How did the instructors find out about MESL?
• How much technological experience do the instructors have?
• Have the instructors ever used digital teaching tools before? If so, what, and how?
• What are the instructional tools the professors currently use?
• What are the benefits of using these tools? The drawbacks?
• How do the instructors intend to use MESL? In the classroom? Outside of the classroom?
• What do the professors perceive the benefits of MESL to be? The drawbacks?
• In a perfect world, what do the instructors think MESL should include or be able to do?"

McMullen, s. (2003). Usability Testing and Library Web Site Redesign at Roger Williams University. Retrieved June 16, 2003, from: http://gamma.rwu.edu/users/smcmullen/usable.html#Usability%20Testing.

• Informational Gathering:
  Seeking Answers to:
  o How usable is the current RWU Library web site?
  o Are students using the library web site for research?
  o How can the library web site make the information seeking process more intuitive and productive?
  o What are users thinking as they attempt to find information on the web?
  o How are they finding information for research?
  o Do students know what they are looking for?
  o Are users aware of the differences between traditional library tools, such as online catalogs & indexes, and web search engines?
  

MIT libraries (2003). Web site usability test: User survey. Retrieved June 16, 2003 from: http://macfadden.mit.edu:9500/webgroup/usability/results/materials.html.

• Test Questions:
  (pdf format: requires Acrobat Reader)
  · Test A: observer worksheets
  · Test A: large-size questions
  · Test B: observer worksheets
  · Test B: large-size questions
  

NISO/DLF Serials Survey. Retrieved April 6, 2003, from http://www.niso.org/survey/serials.cfm

Questionnaires about digital library test collections. Retrieved April 6, 2003, from http://www.sztaki.hu/delos_wg21/q_expl.html.

"Two similar questionnaires are presented here. The one named "available digital libraries and test collections" asks for the type of collections that are already available and could be used for research in the field of digital libraries. In contrast, the other questionnaire "desired digital library test collections" addresses the characteristics of collections that are required for future evaluations in this field." (We need to know another test collection is establishing")

Questionnaire for users of the Oxford University Theology Faculty Digital Library web site. Retrieved April 6, 2003, from http://resources.theology.ox.ac.uk/library/content/feedback.html

Reference Electronic Database Questionnaire. Retrieved April 6, 2003, from http://www.library.sjsu.edu/test/testeval.htm

Software usability measurement inventory. Retrieved April 6, 2003, from http://www.ucc.ie/hfrg/questionnaires/sumi/uksample.pdf.

Swan, M. (2002). Variations2: IU digital music library first-round usability test report. Retrieved June 16, 2003, from: http://variations2.indiana.edu/pdf/v2v1-early-test.pdf.

Testing for Usability in Design of a New Information Gateway: Questions for
Testing Generation 2. Retrieved June 16, 2003, from: http://www.library.arizona.edu/library/teams/access9798/usability_studies/questions.htm.

Some questions extracted from articles or projects

Crum, J., Judkins, D., & Zeigen, L. (2003). A tale of two needs: Usability testing and library orientation. Computer in Libraries, 23(1), 22-25. (From IRIS).

Hill, L. L. (2001). Alexandria digital library: User evaluation studies and system design. Journal of the American Society for Information Science, 51 (3), 246-259.

"User evaluation findings It is useful to present the results of the ADL user evaluations in relation to the following questions:

• What have we learned about our users?
• What have we learned about the evaluation and user study approaches?
• What have we learned about the ADL interface?
• What have we learned about the functionality and content of ADL?" (p. 254)

Jones, C., & Sumner, T. (2003). Evaluation of the National Science Digital Library. Retrieved April 6, 2003, from http://www.uclic.ucl.ac.uk/annb/DLUsability/JonesSumner5.pdf

In an attempt to take preliminary steps to understand these complex usability issues, the NSDL Evaluation Working Group is conducting a pilot study to examine the following four questions:

• How are people using the libraries?
• How are collections growing/changing?
• How well are the distributed library building and governance processes working?
• What is the time and effort required, both centrally and at distributed sites, to participate in this evaluation? Do the results justify this effort?" (p.5)

Marchionini, G. (2000). Evaluating digital libraries: A longitudinal and multifaceted view. Library Trend, 49(2),

…Digital library success is aided by:

• Clear missions;
• strong leadership and a strong talent pool;
• good technical vision and decisions;
• quality content and data management;
• giving users multiple access alternatives; and ongoing evaluation effort.