Design, query, and evaluate information retrieval systems
Understanding the Competency
Information retrieval systems form the foundation of how information is organized, discovered, and accessed in digital environments. Designing, querying, and evaluating these systems requires an understanding of both technical structures and user behavior. This competency emphasizes the ability to create systems that support efficient search and discovery while ensuring that users can locate relevant information with minimal barriers.
Information retrieval systems function by organizing data in ways that allow users to locate specific items or concepts through structured queries. These systems rely on metadata structures, indexing practices, and search logic that translate human information needs into machine-readable forms. Manning, Raghavan, and Schütze (2008) describe information retrieval as the process of “finding material…of an unstructured nature…that satisfies an information need from within large collections,”. Effective systems therefore depend upon thoughtful design choices that determine how information is structured, categorized, and retrieved.
Designing an information retrieval system involves several interconnected stages. First, developers must define the scope and purpose of the collection, including the intended user group and the types of information the system will contain. Next, the data structure must be created, including fields, controlled vocabularies, and indexing rules that support consistent data entry. Query design then allows users to express information needs through search interfaces, filters, and keywords. Finally, evaluation ensures that the system functions as intended by assessing usability, accuracy, and retrieval effectiveness.
These processes reflect broader principles within information science related to information organization, metadata design, and user-centered system development. As Baeza-Yates and Ribeiro-Neto (2011) note, successful information retrieval systems "retrieve all the documents which are relevant to a user while retrieving as few non-relevant documents as possible.” In professional information environments such as libraries, archives, and digital asset management systems, retrieval design directly determines whether collections are discoverable and usable.
Why It Matters to the Profession
As collections continue to grow in size and complexity, the ability to design and evaluate information retrieval systems has become increasingly important for information professionals. Without effective retrieval systems, even well-organized collections can become difficult to navigate, limiting their usefulness and accessibility.
Libraries, archives, and cultural institutions increasingly rely on digital platforms to provide access to their collections. These platforms require structured metadata, searchable interfaces, and retrieval algorithms that support discovery across large datasets. Modern information retrieval systems must address both the organization of information, and the ways in which users interact with search systems. This makes system design a critical component of information service delivery.
As previously mentioned, retrieval systems rely on metadata structures and indexing practices to connect users with relevant information. In archival and digital asset management environments, these design decisions shape how materials are discovered and interpreted. Metadata fields, indexing choices, and search filters influence how users explore collections and whether they are able to locate materials relevant to their needs. Hjorland (2011) suggests that retrieval systems reflect broader knowledge organization principles that shape how information is categorized and understood.
From a professional perspective, this competency highlights the growing importance of technical literacy within the information field. Information professionals are increasingly expected to collaborate with developers, design metadata schemas, and evaluate search systems to ensure that they support user needs. These responsibilities extend beyond simply organizing materials, but involve shaping the technological infrastructure that enables access to information.
Awareness of Information Retrieval Systems in an Information Environment
As previously discussed, information retrieval systems depend on the integration of metadata, indexing structures, and query interfaces. Within an information environment, these components function together as part of a broader information system that manages the storage, organization, and retrieval of data.
Metadata plays a particularly important role in this process by providing structured descriptions that allow systems to connect user queries with relevant records. Consistent metadata enables systems to interpret search terms accurately and retrieve materials that match a user’s information need. Consistent and well-structured metadata is essential to ensuring that information systems can accurately retrieve relevant results. Without standardized descriptive elements systems may return incomplete or inconsistent search results, limiting the usefulness of the collection.
Indexing further supports retrieval by linking metadata elements to searchable terms within the system. Through indexing, the system recognizes relationships between user queries and stored records, allowing materials to be retrieved through keywords, filters, or structured search parameters. As previously noted, these structures must be designed with both the content of the collection and anticipated user search behaviors in mind.
The search interface represents the user-facing component of the information system. Interfaces often include keyword search fields, dropdown menus, and filtering tools that allow users to refine their queries. Marchionini writes (1995), “Information seekers must go through a process of clarification to articulate a search request, with the obvious implication that search systems should support iterative and interactive dialogues with users.” Understanding how these systems function within an information environment allows information professionals to design and manage systems that support efficient and meaningful information discovery.
Impact of Information Retrieval Systems on the Organization
Information systems significantly influence how organizations manage, access, and distribute information. Within libraries, archives, and cultural institutions, these systems provide the infrastructure that enables collections to be organized, searched, and shared with users. The design of an information system therefore has a direct impact on how effectively an organization can support information discovery and knowledge access.
As previously discussed, retrieval systems depend on structured metadata, indexing rules, and search interfaces. When these elements are implemented effectively, they create an infrastructure that supports consistent workflows and reliable access to information. For organizations such as libraries, archives, and cultural institutions, well-designed information systems improve operation efficiency by standardizing how information is entered, stored, and retrieved. Consistent metadata fields and indexing rules reduce duplication, support data integrity, and ensure that records remain searchable as collections grow.
Information systems also shape how users interact with collections. The design of search interfaces and retrieval functions available determine how easily users can locate and explore available materials. If a system is poorly designed or lacks sufficient metadata structure, users may struggle to locate relevant information even when it exists within the collection. As Borgman (2007) observes, digital information infrastructures influence not only how information is accessed but also how knowledge is organized and understood within institutions.
In addition to improving access, effective information systems support long-term sustainability within organizations. By maintaining consistent data structures and retrieval processes, organizations can ensure that their collections remain accessible as technologies evolve and datasets grow larger. This is particularly important in digital environments, where information must remain usable across changing platforms and systems.
For information professionals, the ability to design and evaluate information systems contributes directly to organizational success. By developing systems that support reliable retrieval and scalable data management, professionals help organizations ensure that their collections remain accessible, usable, and relevant over time.
Evidence 1
In INFO 202: Information Retrieval System Design (Spring 2023), we were split into groups to develop an alpha prototype of an information retrieval database focused on gemstones. While this was a collaborative group project, this artifact demonstrates my ability to participate in the design of an information retrieval system by contributing to its metadata structure, indexing rules, and overall organization.
Link: https://drive.google.com/file/d/1d-u92kai42F6G2eLjg7gnYMKgcSBNpvR/view?usp=sharing
Description of the Artifact
The purpose of the gemstone database was to create a searchable system that provides users with accessible information about commonly known gemstones while emphasizing sustainability and ethical sourcing. The system was designed for users who may not have extensive knowledge of gemstones, but are interested in learning about their characteristics and potential sustainable jewelry options.
The database was structured using multiple metadata fields that allowed users to search for gemstones using specific attributes. These attributes included gemstone name, hardness based on the Mohs scale, color variations, mineral composition, origin, mining methods, and sustainability indicators. Together, these fields allowed users to construct targeted queries and retrieve information based on both physical characteristics and ethical consideration.
My contributions focused primarily on the development and refinement of several metadata elements within the system. I was responsible for defining the “Name of Gemstone,” “Can it be sustainable?,” and “Where to find sustainable jewelry?” fields. These fields required establishing indexing rules to ensure consistent data entry across the database. For example, the sustainability field required determining whether a gemstone could be ethically sourced based on criteria such as certification, traceability, or lab-grown production. The sustainable jewelry field required identifying a single brand that met these sustainability criteria.
In addition to defining these fields, I contributed to the development of indexing rules that guided how catalogers should input information into the system. These rules addressed issues such as formatting, capitalization, and acceptable field values to ensure consistency across records. Establishing these rules was an important step in supporting accurate retrieval, as inconsistent data entry can significantly affect search results.
I also contributed to the research and creation of several gemstone records, including Aquamarine and Sapphire, which were used as part of the database’s initial dataset. This process required identifying reliable sources for gemstone characteristics, mining practices, and sustainability considerations. The information collected during this research was used to populate the database and test the functionality of the search interface.
Finally, I served as the primary editor for the project documentation, ensuring that the statement of purpose, indexing rules, and metadata descriptions were clearly written and aligned with the system’s design. I was also responsible for submitting the completed alpha prototype and later posting the beta prototype documentation and database links for peer evaluation.
Justification and Connection to the Competency
This artifact demonstrates the design component of the competency by showing my involvement in developing the structure of an information retrieval system. Designing the database required identifying the types of information users would likely search for and translating those needs into structured metadata fields. My work defining the gemstone name and sustainability-related fields contributed to the overall schema that allowed users to query the database effectively.
The project also illustrates the importance of indexing rules in supporting retrieval accuracy. By developing clear instructions for how information should be entered into specific fields, the project ensured that records would remain consistent and searchable. Without standardized metadata practices, users might encounter incomplete or inconsistent search results, reducing the system’s effectiveness.
My role in researching and populating records further demonstrates how metadata structures function within a retrieval system. Entering records into the database required applying the established indexing rules and verifying that the information aligned with the system’s data structure. This process also served as an initial test of the database’s ability to retrieve relevant results when users queried specific attributes.
Additionally, the project reflects the collaborative nature of information retrieval system development. While different team members were responsible for various fields and components of the system, the final database required integrating these elements into a cohesive structure. Participating in this collaborative process provided experience in designing systems that balance technical organization with user-centered retrieval.
Overall, this artifact demonstrates my ability to contribute to the design of an information retrieval system by developing metadata fields, establishing indexing rules, and supporting the creation of a structured database that allows users to locate relevant information efficiently.
Evidence 2
Related to Evidence 1, this artifact is an evaluation of a prototype information retrieval system that was developed by another group in INFO 202: Information Retrieval System Design (Spring 2023). My primary responsibility included evaluating the database’s indexing rules, entering a plant record into the system, editing the final document, and submitting the completed assignment. This artifact demonstrates my ability to critically evaluate an information retrieval system by examining its metadata rules, identifying inconsistencies between indexing guidelines and database fields, and assessing how these issues affect both cataloging and search functionality.
Link: https://drive.google.com/file/d/1UW_Zx3-jUrN9U-SZgROKs7-mwGl9NzB1/view?usp=sharing
Description of the Artifact
The plant database evaluated in this project was designed to assist new plant owners in selecting plants appropriate for their living environments. The system included metadata fields describing plant characteristics such as light requirements, watering frequency, color, price range, and whether the plant was safe for pets. These attributes were intended to help users search for plants that matched specific environmental conditions and lifestyle considerations.
As part of the evaluation process, each member of the group entered a plant record into the database in order to test how well the system supported data entry and retrieval. I contributed to the Spider Plant record. Entering this record required consulting the database’s indexing rules and identifying appropriate values for each field. This process served as a practical test of the database’s design by revealing how easily catalogers could interpret the rules and apply them consistently.
My primary analytical contribution to the evaluation involved reviewing the indexing rules associated with the database. These rules were intended to guide catalogers in entering information consistently across records. I examined the clarity of the rules, their alignment with the database fields, and whether they adequately accounted for variations in plant characteristics.
During this analysis, I identified several inconsistencies between the indexing rules and the submission form interface. In some cases, the rules referenced dropdown menus or field values that were not available within the database. For example, certain fields instructed catalogers to select from predefined options even though the submission form provided only a text field. This mismatch created uncertainty about how information should be entered and increased the likelihood of inconsistent records.
Additional issues were observed in the categorization of plant attributes. Some fields combined different types of classifications, such as plant life cycles and growth habits, which introduced ambiguity into the indexing process. Other fields lacked sufficient options to accurately represent plant characteristics, requiring catalogers to modify or simplify information in order to fit the available categories.
In addition to my analytical role, I served as the editor for the evaluation document, ensuring that the group’s findings were clearly presented and aligned with the project requirements. I was also responsible for posting the completed evaluation to the course discussion board once the final draft had been reviewed by the group.
Justification and Connection to the Competency
This artifact demonstrates the evaluation component of Competency E by analyzing how effectively an information retrieval system supports both cataloging and search functionality. Evaluating the database required examining the relationship between indexing rules, metadata fields, and user-facing search interfaces to determine whether the system enables accurate information retrieval.
My analysis of the indexing rules highlighted the importance of consistency between system documentation and database design. When indexing rules do not match the available submission form fields, catalogers may interpret instructions differently, resulting in inconsistent metadata across records. These inconsistencies can reduce retrieval effectiveness by limiting the system’s ability to match user queries with relevant results.
The process of entering a plant record also demonstrated how database structures influence the cataloging process. By attempting to apply the indexing rules to a real record, I was able to observe how limitations in field values or unclear instructions could affect the accuracy and completeness of metadata. This type of practical testing is an important step in evaluating retrieval systems, because it reveals issues that may not be immediately visible when reviewing the database structure alone.
Additionally, this project reinforced the importance of user-centered design in information retrieval systems. The database was intended for new plant owners, so the metadata fields and search options needed to align with the types of questions users would likely ask (i.e., whether a plant is safe for pets or how much sunlight it requires.) Evaluating the system from this perspective allowed the group to assess whether the database design supported the information needs of its intended audience.
Overall, this artifact demonstrates my ability to evaluate an information retrieval system by analyzing indexing rules, testing database functionality, and identifying design limitations that affect both cataloging consistency and user retrieval. These evaluation skills are essential for information professionals who work with digital databases, archival systems, and other platforms that depend on structured metadata and reliable search functionality.
Evidence 3
The final piece of evidence is a recorded presentation created for INFO 281: Metadata (Spring 2025), titled Optimizing Metadata for Visual Resources: Why Corporate Archives in Entertainment Should Adopt VRA Core. The presentation examines how the VRA Core metadata standard can improve the description, organization, and retrieval of visual materials within corporate entertainment archives. This artifact demonstrates my ability to analyze and evaluate metadata frameworks that support information retrieval systems, particular in environments managing complex visual media assets.
Link: https://drive.google.com/file/d/1fusPgAvlfqlVLI35uJXgFiJSU8V7hyKp/view?usp=sharing
Description of the Artifact
This presentation explores the challenges that corporate archives in the entertainment industry face when managing visual resources such as concept art, animation cels, production stills, set designs, character models, and promotional materials. These materials often exist in multiple versions and formats throughout the production process, which creates complex relationships between assets that must be documented and maintained.
The presentation begins by outlining the limitations of commonly used metadata standards such as Dublin Core and MARC when applied to visual resources. While these standards are widely used in libraries and archives, they are often too general or bibliographically oriented to describe the complex relationships between visual production assets. As a result, corporate archives may struggle to provide sufficient metadata detail for effective retrieval.
The presentation then introduces VRA Core, a metadata standard developed by the Visual Resources Association specifically for describing works of art and visual media. One of the most important features of VRA Core is its distinction between “work” records and “image” records, which allows archivists to document both the original creative work and the various reproductions or versions associated with it. In entertainment archives this distinction is particularly valuable, because visual assets often exist in multiple stages of production (i.e., sketches, digital renderings, and final promotional images.)
Another major focus of the presentation is how VRA Core supports more detailed descriptive metadata. The schema includes fields that document creators, dates, formats, locations, relationships, between assets, and rights information. These elements enable archivists to capture more contextual information about visual materials, which improves both asset management and search functionality within digital systems.
The presentation also compares VRA Core to other metadata frameworks commonly used in archival and cultural heritage contexts, including Dublin Core, MARC, and CIDOC-CRM. While each schema has its pros, the presentation argues that VRA Core is often more practical for corporate archives. VRA CORE is easier to implement within digital asset management systems, and is specifically designed for visual media. To illustrate the potential impact of VRA Core on information retrieval, the presentation includes a hypothetical case study involving a licensing team searching for variations of a corporate logo.
Finally the presentation outlines an implementation roadmap for organizations interested in adopting VRA Core. This roadmap includes conducting a metadata audit, mapping existing metadata fields to VRA Core elements, integrating the schema into digital asset management systems, and training staff on consistent metadata practices.
Justification and Connection to the Competency
This artifact demonstrates the design and evaluation aspects of Competency E by examining how metadata schemas influence the effectiveness of information retrieval systems. Metadata structures function as the underlying framework that enables information systems to organize and retrieve data. By analyzing the capabilities of VRA Core, the presentation evaluates how different metadata standards support retrieval processes in visual media collections.
The project required examining how metadata fields affect indexing and search functionality within digital asset management systems. For example, the distinction between work and image records in VRA Core creates a more structured way to represent relationships between assets. This structure allows retrieval systems to differentiate between original works and derivative representations, which improves the accuracy of search results when users query visual collections.
The presentation also highlights how metadata design affects the ability of users to locate specific assets within large collections. In corporate entertainment archives, users from multiple departments–including researchers, marketing teams, and licensing staff–may need to retrieve assets based on different criteria. By supporting detailed descriptive fields and structured relationships between assets, VRA Core allows information systems to support more precise queries and more efficient retrieval.
Additionally, the comparison of VRA Core with other metadata standards demonstrates the importance of selecting appropriate metadata frameworks for specific information environments. Different schemas prioritize different types of information, and choosing a standard that aligns with the nature of the collection can significantly improve the functionality of retrieval systems.
This artifact also reflects my professional interest in digital asset management within the entertainment industry. Corporate archives managing film, animation, and comic properties rely heavily on digital asset management systems to organize and retrieve visual materials. Understanding how metadata schemas interact with these systems is essential for ensuring that assets remain discoverable, usable, and properly contextualized.
Overall, this artifact demonstrates my ability to evaluate metadata standards as components of information retrieval systems. By analyzing how VRA Core supports structured metadata, relational asset tracking, and improved search functionality, the project illustrates how thoughtful metadata design can enhance the discoverability and usability of complex visual collections.
Conclusion
Designing, querying, and evaluation information retrieval systems is essential to ensuring that digital collections remain accessible, organized, and meaningful to users. As information environments continue to grow in scale and complexity, the effectiveness of these systems increasingly depends on the thoughtful design of metadata structures, indexing rules, and search interfaces. Through the artifacts presented in this competency, I have demonstrated my ability to contribute to the development and evaluation of information retrieval systems while recognizing how metadata design directly influences the discoverability and usability of information.
The gemstone database prototype illustrates my involvement in the design of an information retrieval system. Contributing to the development of metadata fields, indexing rules, and database records required translating anticipated user needs into structured metadata elements that support effective searching and retrieval. The beta evaluation of the plant database demonstrates the evaluation component of information retrieval systems. By analyzing the relationship between indexing rules, database fields, and submission interfaces, I was able to identify inconsistencies that could affect cataloging practices and retrieval accuracy. The presentation on VRA Core further expanded this understanding by examining how metadata standards influence retrieval systems within specialized information environments. Evaluating the capabilities of VRA Core demonstrated how specialized metadata frameworks can improve the organization, indexing, and retrieval of visual resources within digital asset management systems.
Together, these artifacts demonstrate the interconnected nature of system design, query functionality, and system evaluation. Information retrieval systems are not static. They require ongoing refinement to ensure that metadata remains consistent, search tools remain effective, and collections remain accessible as they evolve. Developing these skills has strengthened my understanding of how information systems function within professional environments.
In my future career, particularly in archives and digital asset management across art and entertainment fields, these principles will be essential for managing complex visual collections. Production assets such as concept art, animation frames, storyboards, and promotional materials depend on well-structured metadata and effective search systems to support collaboration across departments including research, marketing, and licensing. This competency has strengthened my understanding of the relationship between metadata design, system functionality, and information access. By applying the principles of information retrieval system design and evaluation, I will be able to contribute to systems that support efficient discovery and long-term preservation of digital assets.
References
Baeza-Yates, R., & Ribeiro-Neto, B. (2011). Modern information retrieval: The concepts and technology behind search. Addison-Wesley.
Borgman, C. L. (2007). Scholarship in the digital age: Information, infrastructure, and the Internet. MIT Press.
Hjørland, B. (2011). The importance of theories of knowledge: Browsing as an example. Journal of the American Society for Information Science and Technology.
Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to information retrieval. Cambridge University Press.
Marchionini, G. (1995). Information seeking in electronic environments. Cambridge University Press.
