Course Objectives:

Students will investigate the range of evaluative methods used in library and information science to assess institutional, system and service performance. Value issues and external validity issues will be highlighted during this review. Emphasis throughout the course will be on the impact of systems and services to end-users.

Measurement issues will be explored as they relate to evaluation goals and objectives. Students will simulate the design of an evaluation program for targeted users and will analyze the results obtained using quantitative and qualitative research methods. Effective methods for graphing data results will be emphasized for this objective. Computer packages will be used to demonstrate data analysis and the creation of graphic displays. Computer laboratory sessions will be held as needed to supplement the course topics.

Alternative methods to evaluate services and systems will be explored with a concomitant analysis of the advantages and disadvantages of each approach. Specific references will be made to the appropriateness of different evaluative methodologies and criteria.

Course Organization:

1. Introduction to course
   1. Brief history of public library, including issues pertinent to 572
   2. PowerPoint overview of issues
      1. Disruptive technologies
      2. TFG library assumptions
   3. Related issue: institutional performance as a measure of accountability
   4. Purpose of accountability and its role in planning
2. Disruptive technologies vs. Balanced Scorecard. For reference only:
   1. Christensen, Clayton M. The Innovator?s Dilemma; The Revolutionary National Bestseller That Changed The Way We Do Business. New York: HarperCollins (HarperBusiness), 2000. 252 p. ISBN 0-06-662069-4
   2. Kaplan, Robert S.; Norton, David P. The Strategy-Focused Organization; How Balanced Scorecard Companies Thrive in the New Business Environment. Boston, Massachusetts: Harvard Business School Press, 2001. 400 p. ISBN 1-57851-250-6
3. Hernon/Whitman: Delivering Satisfaction and Service Quality. Understanding three essential areas:
   1. Understanding customer service: Hernon/Whitman, Chap. 1, pp. 1-13.
   2. Understanding service quality: Hernon/Whitman, Chapter 2, pp. 14-30.
   3. Understanding customer satisfaction: Hernon/Whitman, Chapter 3, pp. 31-73.
4. Methodological issues in evaluation/performance studies (class 4)
   1. Theoretical & structural issues
   2. Sampling [N.B.: unit of analysis issue for organization vs info retrieval system]
   3. Design: experimental studies; survey research; system designs
   4. Value issues
   5. External validity issues
   6. Outputs, inputs, impacts, etc.
   7. Information retrieval system evaluation
5. Framework for improving ServQual & Customer Satisfaction over time:
   1. Hernon/Whitman, Chap. 4 + Appendices, pp. 43-73.
6. Qualitative measures used in evaluation
   1. Case studies
   2. Observation studies I
   3. Interviewing
   4. Diaries
   5. Talk-aloud protocols
   6. Focus group interviews
   7. Other designs and methodologies
7. Creating a new service model
   • Hernon/Whitman, Chap. 5, pp. 74-92
8. Satisfaction: measurement and evaluation
   • Hernon/Whitman, Chap. 6, pp. 93-116
9. Quantitative measures used in evaluation. Computer applications.
   • Hernon/Whitman, Chap. 7, pp. 117-126
   • Hernon/Whitman, Chap. 8, pp. 127-156
   1. Descriptive statistical measures
   2. Graphic displays showing:
   1. Performance over time
   2. Performance compared to peers
   3. Inferential statistics
   4. Information retrieval system issues.
   5. Census data applications
10. New assumptions and new designs for planning alternative futures
    • Hernon/Whitman, Chap. 9, pp. 157-169