Evaluate Programs and Services Using Measurable Criteria
Understanding the Competency
Competency N contains two requirements that are easy to collapse into one. The first is evaluation. The second is that the evaluation rests on measurable criteria. Keeping them apart is the whole task, because much of what passes for evaluation in practice is only measurement.
Nicholson (2004) draws the distinction precisely, borrowing definitions from Scriven. Measurement is the “determination of the magnitude of a quantity” (Scriven, 1991, as cited in Nicholson, 2004, p. 497), while evaluation is “the process of determining the merit, worth, or value of something” (Scriven, 1991, as cited in Nicholson, 2004, p. 497). For example, counting how many people attended a program is measurement, while deciding whether that attendance was good is evaluation. These are not the same act, and it is important to make that distinction.
What is missing in measurement is a criterion. Nicholson offers the plainest definition of evaluation available, taken from Van House et al. (1990, as cited in Nicholson, 2004, p. 497), which is that evaluation consists of comparing “what is” to “what ought to be.” The criterion is the “ought,” and without it there is nothing for the measurement to be compared against. Nicholson names the practical failure this produces when he observes that “measuring without evaluating is a common problem with automated reporting tools” (2004, p. 502). Any institution with a dashboard has encountered this problem, where data accumulates steadily and nobody ever renders a judgment on it.
The word measurable also needs defining, because it is commonly misread as numerical. A criterion is measurable when it is stated clearly enough that a second person applying it to the same program would reach the same judgment, which means reproducibility is the real test rather than arithmetic. A yes or no check can be perfectly measurable (e.g., does the collection publish an accessibility statement, are transcripts provided, does the search support Boolean operators), and each of those is verifiable by someone else working independently. A number with no stated threshold, by contrast, is not a criterion at all, because nobody can say what value would count as success.
Criteria come from two places. The first is whether I can build them myself, as in a collection development policy or a rubric written for a single project. The second is whether I can adopt them from a professional body, as with the RUSA reference guidelines or an accessibility standard. Criteria I build fit my collection and my users exactly, but they carry no authority outside my institution and nobody can check my judgment against anyone else’s. Criteria I adopt carry professional weight and allow comparison across institutions, but they were written for a general case and may not fit the thing in front of me. Most real evaluation uses both, and the skill lies in knowing which to use where.
Underneath all of this sits a point Nicholson (2004) makes almost in passing, which is that judgment requires a viewpoint. Criteria are never neutral, because they encode somebody’s idea of what good looks like, and whoever sets them decides what will be examined and what will go unexamined. During my MLIS program at San José State University, I developed this competency through work that required me to state what I was judging against before I judged, to apply professional standards I did not write, and to discover that a plan can look complete on paper and fail the moment anyone tries to execute it.
Measurement, Criteria, and Evaluation
Nicholson (2004) supplies the fullest framework in the literature for thinking about this. His argument begins from systems theory, holding that a library is an interconnected system and that evaluating one component in isolation will mislead. Most evaluation, he observes, happens because a problem forced it, which means it tends to be reactive and narrow. He compares this to emergency medicine, where patients wait until symptoms become unbearable, and argues for the equivalent of preventive care.
His measurement matrix crosses two dimensions. Perspective runs between internal, meaning the view of the library and its staff, and external, meaning the view of the user. Topic runs between the library system itself and the use of that system. The result is four quadrants. The internal view of the library system asks what the system consists of and compares its components against a standard. The external view of the library system asks how effective the system is, through usability and relevance from the user’s position. The external view of use asks how useful the library was, through what users report gaining and what they later cite. Lastly, the internal view of use asks how the system was actually manipulated, through transaction logs and recorded interactions. Nicholson’s conclusion about this matrix is the one most worth carrying forward, which is that “there is not a single measure that can be taken that represents the library” (2004, p. 501).
Nicholson (2004) then adds a second structure, which is a pyramid of evaluation viewpoints rising from the measurements themselves through the evaluation criteria and then upward through users, library personnel, and finally decision-makers such as funding agencies and policy makers. Each level evaluates using everything beneath it. The point of the pyramid is that the same criterion produces different judgments depending on who applies it, and that a measure like cost-benefit means something quite different to a user, a librarian, and a funder.
Bruce and Hillmann (2004) address the same problem inside one domain, which is metadata quality. Their chapter opens by admitting that quality is difficult to define and that we tend to know it when we see it. They identify seven characteristics of quality metadata, which are completeness, accuracy, provenance, conformance to expectations, logical consistency and coherence, timeliness, and accessibility, adapted from the Statistics Canada Quality Assurance Framework by way of Paul Johanis.
Two things in that chapter matter for this competency. The first is their claim, drawn from the success of cooperative cataloging programs, that “quality is quantifiable and measurable, and to be effective, enforcement of standards of quality must take place at the community level” (Bruce & Hillmann, 2004, p. 240). Criteria work when a community agrees to them, rather than when one institution invents them privately. The second is the structure of their Table 15-1, which runs three columns across, naming a quality measure, then a quality criteria question, then a compliance indicator. For completeness, the question is whether the element set completely describes the objects, and the indicator is the application profile and the documentation. That three-part move is what makes it something checkable, because it names both the question and the evidence that answers it.
Bruce and Hillmann (2004) are also honest about where they stop. They describe their seven dimensions as “places to look for quality in collection-specific schemas and implementations, rather than as checklists or quantitative systems suitable for direct application” (p. 242). They deliberately decline to write the checklist, on the grounds that no single one would hold across domains, media types, and funding levels.
The Digital Library Federation Assessment Interest Group (DLF AIG) Metadata Assessment Working Group (n.d.) picks up where they left off. Their Metadata Quality Benchmarks organize criteria into the same seven categories the Bruce and Hillmann framework established, and split each category into technical specifications and metadata values. Under accuracy, for example, sit criteria such as whether validation measures exist for record structure and values, and whether default values are correct. The working group is careful about the limits of this, noting that individual criteria “could be used to create measurable benchmarks depending on the priorities and technical capabilities of a specific digital collection,” rather than offering universal thresholds. Even the group doing the operationalizing declines to set the numbers on everyone else’s behalf.
Conway (2010) shows what happens when a field lacks agreed criteria altogether. His article frames four dilemmas facing preservation, covering environmental storage, quality, audiovisual materials, and an expertise gap. The quality dilemma is the relevant one here. He traces a field that cannot settle whether preservation quality means full information capture to a technical specification or fitness for a particular use. He notes that quality checking in digital repositories is largely a matter of inspection rather than validation against standards. He names the absence of standard quality criteria as the gap, and he does not fill it. His warning is that compromises on quality may raise workflow productivity at the cost of the overall use value of the digital products.
Where Criteria Come From
The professional literature supplies criteria at very different levels of specification, and knowing where a given standard sits on that range is itself part of this competency.
At one end sit frameworks that name principles without supplying measures. The National Information Standards Organization (NISO, 2007) published A Framework of Guidance for Building Good Digital Collections, which organizes criteria for goodness around collections, objects, metadata, and initiatives. Its six metadata principles state that good metadata conforms to community standards appropriate to the materials and users, supports interoperability, uses authority control and content standards to describe and collocate objects, states conditions and terms of use clearly, supports long-term curation and preservation, and is itself an object requiring authority, authenticity, archivability, persistence, and unique identification. On evaluation the framework is explicit about its own limits, stating that “the criteria, methods, and metrics for evaluating collections will vary by the objectives of the collections” (NISO, 2007, p. 19).
The Reference and User Services Association (RUSA, 2023) sits in a similar place, and it is worth examining closely because it is the example most often given of ready-made criteria. The 2023 revision defines six areas of observable behavior, which are inclusion, approachability, engagement, searching, evaluation, and closure. Within each area it lists specific actions, such as acknowledging a waiting user promptly, restating a question to confirm understanding, asking what the person has already tried, and confirming that the need was met before closing. These are genuinely observable, which is what makes them usable as criteria at all. What the document does not contain is an instrument. There is no rubric, no scale, no sampling guidance, and no procedure for self-assessment or peer review. The only statement about evaluative use appears in the document history, which notes that the guidelines have been used to assist in the training, development, and evaluation of library workers who provide information services directly to users. That is a description of how the profession has used them rather than a method the profession supplies. Additionally, one caution applies to the section titled evaluation, which concerns helping users judge whether information is accurate, authoritative, current, and relevant. It is not about assessing staff performance.
Federal requirements stop in much the same place. The Institute of Museum and Library Services (IMLS, 2022) requires every state library agency to evaluate its five-year plan independently, and prescribes the questions the resulting report must answer in order. On measurement, though, the guidance offers a single three-level scale, asking agencies to rate each goal as achieved, partly achieved, or not achieved. There are no numeric targets, no required baselines, and no standard metrics, because each agency is judged against the goals in its own plan. Even a federal evaluation mandate declines to set the yardstick.
At the far end sits the one case where the criteria really are fully specified. The Public Library Association’s (PLA) Project Outcome, now redistributed as Outcome Measurement for Public Libraries, provides nine standardized surveys measuring four patron outcomes, which are knowledge, confidence, application, and awareness (American Library Association [ALA], 2025). Over 10 years, more than 2,600 libraries collected close to 500,000 patron responses through more than 38,000 surveys (ALA, 2025). In August 2025, PLA retired the online platform and moved the instruments to free static tools for local use (PLA, n.d.). Standardization at that scale is what makes comparison across institutions possible, and it is rare.
Two further sources add more depth to the literature. Manis and Wilde (2024) examine the Library of Congress, which recorded 151.6 million visits and 520.3 million page views in 2022, and argue that provenance principles and metadata schemas determine whether context survives into a digital collection. Their focus is Library of Congress Subject Headings, and their sharpest illustration comes from Cara Finnegan, who could not locate a photograph of a man she took to be a farmer or sharecropper, because the image had been catalogued under the heading “shack.” Their broader point is that criteria embedded in a descriptive standard decide what becomes findable and what does not, and that standards built for text serve visual materials badly. Criteria are not only a tool for judging quality, but also a place where bias becomes structural.
Strommer (2023) supplies a method rather than a standard. Her argument is that a written plan is not the same thing as a working plan, and that the way to find out which one you have is a tabletop exercise. She defines this as a “facilitated discussion, not physical practice,” built around a hypothetical emergency in which participants talk through their roles. Its purposes include familiarizing staff with the plan, practicing in a low-stakes setting, finding the plan’s strengths and weaknesses, and identifying knowledge gaps. Her case study at the University of Illinois Library ran for three hours with roughly 20 disaster team members, two facilitators, two recorders, and a recovery vendor, and it exposed gaps in the public communication plan that reading the document had not revealed.
Why It Matters to the Profession
Libraries, archives, and museums are continuously required to say whether they are any good. Funders ask it, governing boards ask it, and the public asks it indirectly by showing up or staying away. The answer always rests on a claim about merit, and a claim about merit always rests on a criterion, whether or not anyone says so out loud.
When the criterion stays unstated, two things go wrong. The first is that evaluation turns into advocacy, because an institution reports the measures that look best and omits the ones that do not. Since no standard was declared in advance, nobody can demonstrate that the report is incomplete. The second is the problem Nicholson (2004) identifies with automated reporting, where systems generate enormous quantities of data without anyone having decided what any of it would need to show in order to count as success.
Stating criteria in advance is what prevents both. It commits an institution to a standard before it knows whether it will meet that standard, which is uncomfortable and is exactly the point. Additionally, it makes an unflattering result usable rather than merely embarrassing, because a program that falls short of a stated criterion has produced information about where to improve, while a program with no criterion has produced nothing anyone can act on.
There is also an equity dimension here. Criteria determine what gets examined, which means a set of criteria covering completeness, accuracy, and interoperability will reveal nothing about whether a collection describes marginalized people adequately, because that question was never asked. The omission presents itself as neutrality while functioning as a verdict. Choosing criteria is therefore a decision about what an institution is willing to find out about itself.
Evidence 1
The first piece of evidence demonstrating my mastery of Competency N is my metadata evaluation of the Edison Company Motion Pictures and Sound Recordings collection at the Library of Congress, completed in INFO 281-10: Metadata under Dr. Mary K. Bolin and dated May 4, 2025. It was solo work.
Description of the Artifact
The paper is a structured evaluation of a federal digital collection. It opens by establishing the collection’s origin, governance, and funding, tracing the holdings back through the American Memory project and identifying the Motion Picture, Broadcasting, and Recorded Sound Division as the responsible body. It then describes the contents, which comprise more than 340 motion pictures and 80 sound recordings dating from 1891 to 1918, and the metadata standards in use, including MODS, METS, Library of Congress Subject Headings, and the Name Authority File.
A literature review follows, covering metadata principles in audiovisual archives and the particular difficulty of describing formats bound to obsolete hardware. It draws on Harvey, Casey, Baca, Conway, and Gilliland, and it incorporates Manis and Wilde on the way established frameworks can obscure contributors within the historical record.
The evaluation proper begins with a labeled Evaluation Criteria section naming five criteria, which are metadata usage and completeness, content richness and historical value, navigation and user interface, search functionality, and display and accessibility. The results section is then organized under those same five headings, with findings reported against each criterion in turn.
The paper closes with a comparative case study of the UCLA Film and Television Archive’s digital collections, examining differences in metadata practice, interface design, and accessibility between the two institutions.
Justification and Connection to the Competency
This artifact demonstrates the core structural requirement of the competency, which is that criteria are declared before judgment is passed and then actually used. The criteria appear in a section of their own, before any findings, and the results section repeats the same five headings in the same order. That arrangement makes the evaluation auditable, because a reader can see what I was judging against and check whether I judged against it consistently. Most evaluations do not survive that test, since their criteria have been reconstructed from their conclusions.
The artifact also demonstrates that I can work with criteria from both sources at once. The five criteria are mine, built for this collection, but the evaluation also measures the collection against standards I did not write. The metadata assessment tests conformance to MODS, METS, and authority control practice, while the accessibility assessment tests against WCAG and universal design principles. The same paper therefore applies self-constructed criteria to content richness and interface design, and adopted professional criteria to description and accessibility.
Additionally, my self-built criteria converged independently on an established standard I had not cited. My finding that performer names are inconsistently encoded, with some entries under authority control and others entered as plain text, is a failure against NISO’s (2007) third metadata principle, which requires authority control and content standards to describe and collocate objects. My finding that the records lack administrative metadata such as nitrate film base, recording speed, and links to conservation notes is a failure against the fifth principle, which requires that metadata support long-term curation and preservation. Arriving at the same tests independently is evidence that the criteria were defensible rather than arbitrary.
The findings themselves are also more measurable than the criteria section admits. Each one is a reproducible check. Is a performer name linked to an authority record or entered as free text? Does an item include a transcription? Does it include synchronized captions? Does the search support Boolean operators? Can results be faceted by duration, recording method, or performer? Does the institution publish an accessibility statement? Every one of those has an answer another evaluator would reach independently, which is the sense in which a criterion is measurable.
The UCLA comparison adds a third source of criteria, which is benchmarking against a peer institution. Judging the Edison collection against what a comparable archive actually achieves is different from judging it against a principle, because it establishes that the gap is closable rather than theoretical. UCLA’s use of EAD alongside MODS, its incorporation of locally specific subject terms, and its published accessibility documentation all demonstrate that the standard I was holding Edison to is already being met elsewhere.
More broadly, this artifact demonstrates that I can evaluate an institutional resource systematically rather than impressionistically, and that I can locate my own criteria within the professional standards that already exist.
Evidence 2
The second piece of evidence demonstrating my mastery of Competency N is the Strategic Plan for the Hawaii State Library, completed in INFO 204: Information Professions under Professor Sean Gaffney and dated December 6, 2023. This was group work by a team of six.
Description of the Artifact
The plan is a full strategic planning document for the Hawaii State Public Library System. It opens with a historical background tracing the system from the Honolulu Library and Reading Room Association in 1879 through the establishment of the only statewide public library system in the United States. An environmental scan follows, covering demographics, funding structure across state general funds, capital improvement funds, special funds, and federal Library Services and Technology Act funds, and the pressures created by the 2023 Maui wildfires and by competition with the Hawaii Tourism Authority for state money.
A SWOT analysis synthesizes that material into strengths, weaknesses, opportunities, and threats. The plan then sets out four goals, covering literacy, the needs of the homeless population, the needs of the aging community, and marketing. Each goal carries its own Strategies subsection and its own Assessment subsection.
An appendix documents the division of labor member by member. I held a research role for the historical background and environmental scan, I wrote the marketing goal along with its strategies and assessment in full, and I co-edited the final document with one other member, taking responsibility for the overall presentation.
Justification and Connection to the Competency
I selected this artifact, because it demonstrates the forward-looking half of this competency, which is the construction of criteria before a program exists. The Edison evaluation judged something already built, while this plan specifies in advance how anyone would later know whether four proposed programs had worked. That is the harder problem, because the criteria have to be written without knowing what the results will be.
The marketing goal is mine, and its assessment plan is the most specified of the four. It names instruments rather than intentions, identifying W3Counter and StatCounter for tracking online engagement. It builds comparison into the design through two separate before and after measures, surveying community awareness before and after the branding rollout and counting event attendance before and after the marketing implementation. Additionally, it adds an attribution question to the program surveys, asking patrons how they heard about the event.
That last measure is the one worth drawing attention to. Attendance and attribution answer different questions, since attendance tells you whether the goal was met while attribution tells you which channel produced the result, which is what you actually need in order to act on the finding. Separating the two is the kind of move Nicholson (2004) argues for when he insists that no single measure represents a system and that different evaluation criteria draw from different quadrants of measurement. A marketing program evaluated only by turnout would tell the library it had succeeded without telling it what to do again.
My section also applies an adopted external standard. The website redesign strategy calls for running a WAVE report to identify missing alternative text and contrast errors, and WAVE tests against WCAG, so that part of the plan commits in advance to a criterion set by a standards body rather than by the team. The same pattern appears here as in the Edison evaluation, with accessibility judged against an adopted standard while the remainder of the plan is judged against criteria we wrote ourselves.
The plan also sits inside an evaluation cycle rather than beginning from nothing. It builds on an external evaluation of the system’s previous five-year plan, conducted by OhioNet, which identified low awareness of online databases and low participation in online learning tools. That report exists because IMLS (2022) requires every state library agency to evaluate its five-year plan independently. Our plan therefore inherits criteria from a federal requirement, applies findings from a prior evaluation, and sets new criteria for the next cycle, and understanding that chain is part of understanding how evaluation functions in publicly funded library systems.
The limitation is consistent with the one in Evidence 1. The measures name what will be tracked, but not what would count as success. Measuring attendance before and after does not say how large an increase would mean the strategy worked, or over what period. IMLS itself stops at a three-level scale of achieved, partly achieved, and not achieved. So the plan is not unusual in this respect, but a target and a baseline would have made it stronger. Project Outcome shows what the fully specified alternative looks like, with standardized instruments measuring defined patron outcomes across thousands of libraries.
This was group work, so I want to be precise about scope. The literacy, homelessness, and aging goals were written by Ramirez, Villa, and Avila respectively, and their assessment plans belong to them. I am claiming the marketing goal, the research contributions to the first two sections, and the editing of the final document, all of which the appendix documents, and which was written at the time rather than reconstructed afterward.
More broadly, this artifact demonstrates that I can build measurable criteria into a program at the planning stage, inside a real institution facing real funding constraints, and as one member of a team whose individual contribution can be identified.
Evidence 3
The third piece of evidence demonstrating my mastery of Competency N is my Preservation Management and Disaster Response Plan, completed in INFO 259: Preservation Management under Dr. Jessica Bushey and dated April 9, 2024. It was solo work.
Description of the Artifact
The paper evaluates the disaster plan of the Archives of the Archdiocese of Seattle by running a simulated electrical fire against it and documenting what happens.
The simulation proceeds on a timeline. An outlet overloads at 2:00 p.m. on a summer afternoon in the staff offices, smoke spreads, and named staff execute the plan’s evacuation procedures in sequence. The paper works through the roles the plan assigns, the decision about whether to attempt extinguishing the fire, the closing of fire doors, the collection salvage priority list handed to the fire chief, and the selection of an off-site location.
After the fire is out, the paper moves to damage assessment and salvage. It assigns the response and recovery roles the plan specifies, then works through material-specific salvage decisions. Wet film goes to a restoration lab for rewashing within 72 hours. Photographic albums with limited water exposure are air dried, while those with substantial damage are frozen. Print photographs are triaged by process, with albumen, collodion, silver gelatin, and photomechanical prints air dried, carbon prints and Woodburytypes frozen immediately, and dye transfer prints separated and air dried as a priority. Soot removal uses a HEPA filter vacuum followed by soot sponges, drawing on guidance from the American Institute for Conservation.
The conclusion identifies three failures in the plan and recommends a remedy for each.
Justification and Connection to the Competency
This artifact demonstrates a kind of criteria-based evaluation the other two cannot, which is evaluation by execution. The criterion applied here is operational adequacy, meaning whether the plan can be carried out, step by step, under the conditions it was written for. That question cannot be answered by reading, and nothing in the document announces its own failures, since they only appear when somebody tries to use it.
Strommer (2023) describes exactly this method. A tabletop exercise is a facilitated discussion built around a hypothetical emergency, in which participants talk through their roles in order to familiarize themselves with the plan, find its strengths and weaknesses, and identify knowledge gaps. What I produced is a written version of that exercise, and the findings are the kind Strommer predicts it will surface.
The first failure is structural. The plan designates a building as the archives off-site command center, and in a fire scenario that same building is under evacuation, yet the plan names no alternative. The contingency the document provides is unavailable precisely when it is needed, which is a single point of failure visible only when the scenario is run.
The second failure is an obsolete standard. The plan relies on Halon 1301 as its fire suppression agent, and Halon 1301 was identified as an ozone-depleting substance with production ceasing at the beginning of 1994. Thirty years on, recharging the system after a discharge would be prohibitively expensive if it were possible at all. Here I measured the plan against an external criterion the plan’s authors had not kept up with, drawing on National Archives guidance on fire suppression.
The third failure is the sharpest, because it is a scope error at the center of the document. The plan’s salvage recommendations address water damage only, with no guidance for soot, none for smoke, and none for the effects of halon exposure. This is a fire response plan whose salvage guidance does not cover fire. I found it only because the simulation forced me to look for instructions that were not there, and I had to go outside the plan, to the American Institute for Conservation and the National Archives, in order to supply what the plan omitted.
Each of those three findings illustrates a different relationship between criteria and evaluation. The first applies a criterion of internal consistency, asking whether the plan contradicts itself. The second applies a criterion of currency, asking whether the plan reflects standards still in force. Lastly, the third applies a criterion of coverage, asking whether the plan addresses the scenario it exists for.
One limitation is worth stating, which is that I did not declare my criteria in advance in this paper, as the Edison evaluation did. The criteria are visible in the findings but they remain implicit, and a reader has to infer them, whereas writing them down first would have made the evaluation reproducible rather than only persuasive.
More broadly, this artifact demonstrates that I understand evaluation as something performed rather than only written, and that I can apply professional conservation standards as criteria when the plan in front of me falls short of them.
Conclusion
Taken together, these three artifacts show that I can evaluate against criteria in three different relationships to time. The Edison evaluation applies criteria to something already built, declaring the standard in advance and organizing the findings against it. The Hawaii plan builds criteria into programs that do not yet exist, specifying instruments and before and after comparisons so that success can be recognized later. The Seattle simulation tests a finished plan by executing it, applying criteria of consistency, currency, and coverage that only surface under use. Across all three I used self-constructed criteria where none existed and adopted professional standards where they did.
In my future career I expect to apply this competency in archival, media, and cultural heritage settings, as well as in the corporate asset management work I do now. Platforms have to be judged before they are bought and again after they are implemented. Programs funded by grants have to state their intended outcomes before they begin and report honestly against them afterward. Collections have to be assessed for whether their description actually makes them findable, and for whom. In every one of those situations the useful question is the same, which is what would have to be true for this to count as working, and who decided that.
Standards and measures keep developing, so staying current will remain part of my professional practice. I will follow the Digital Library Federation Assessment Interest Group’s Metadata Assessment Working Group, which maintains the Metadata Quality Benchmarks cited here and continues to develop operational criteria for digital collections. I will use the Public Library Association’s Outcome Measurement resources, which provide the standardized survey instruments that represent the most fully specified criteria currently available to libraries. I will monitor the National Information Standards Organization, which issues and revises the recommended practices that define good digital collections. Finally, I will follow the American Institute for Conservation and its National Heritage Responders program, which publishes the salvage and recovery guidance I had to draw on when the Seattle plan proved incomplete. Continued engagement with these bodies will keep the criteria I apply aligned with the ones my field actually recognizes.
Note on the Use of Generative Artificial Intelligence
Claude (Anthropic, 2026) was used in preparing this essay to help identify relevant scholarly literature, to suggest the essay's organizational structure, and to check grammar and APA formatting. All writing, evidence selection, and analysis are my own.
References
American Library Association. (2025, August 19). PLA celebrates Project Outcome success, transitions resources for free, ongoing use. https://www.ala.org/news/2025/08/pla-celebrates-project-outcome-success-transitions-resources-free-ongoing-use
Anthropic. (2026). Claude (Opus 5) [Large language model]. https://claude.ai
Bruce, T. R., & Hillmann, D. I. (2004). The continuum of metadata quality: Defining, expressing, exploiting. In D. I. Hillmann & E. L. Westbrooks (Eds.), Metadata in practice (pp. 238–256). ALA Editions.
Conway, P. (2010). Preservation in the age of Google: Digitization, digital preservation, and dilemmas. The Library Quarterly, 80(1), 61–79. https://doi.org/10.1086/648463
Digital Library Federation Assessment Interest Group Metadata Assessment Working Group. (n.d.). Types of quality metrics. Metadata Quality Benchmarks. Retrieved November 7, 2026, from https://dlfmetadataassessment.github.io/MetadataQualityBenchmarks/quality-metrics.html
Institute of Museum and Library Services. (2022). Guidelines for IMLS Grants to States five-year evaluation, 2023–2027. https://www.imls.gov/sites/default/files/2022-12/2023-2027_fiveyearevaluationguidelines.pdf
Manis, K., & Wilde, P. (2024). Locating yourself in the historical record: Challenges of provenance and metadata schemas in the Library of Congress’s digital materials. Archival Science, 24, 897–922. https://doi.org/10.1007/s10502-024-09465-7
National Information Standards Organization. (2007). A framework of guidance for building good digital collections (3rd ed.). https://www.niso.org/sites/default/files/2017-08/framework3.pdf
Nicholson, S. (2004). A conceptual framework for the holistic measurement and cumulative evaluation of library services. Proceedings of the ASIST Annual Meeting, 41(1), 496–506. https://doi.org/10.1002/meet.1450410158
Public Library Association. (n.d.). Outcome measurement for public libraries. American Library Association. Retrieved November 7, 2026, from https://www.ala.org/pla/data/outcomemeasurement
Reference and User Services Association. (2023). Guidelines for behavioral performance of reference and information service providers. American Library Association. https://www.ala.org/rusa/resources/guidelines/guidelinesbehavioral
Strommer, S. (2023, August 17). Practicing your plan before the disaster strikes [Conference presentation]. IFLA World Library and Information Congress 2023 Satellite Meeting: Strengthening Libraries’ Emergency Preparedness and Response. https://repository.ifla.org/items/50f17828-7c03-4e22-ad10-3db46bc73c82
