Auditing algorithms in commercial discovery tools
"Library search tools are littered with algorithms that determine what a search ""means"" and what items are ""relevant,"" among other things. Evaluating these algorithms is hard, because their workings are unknown. The algorithms are the major intellectual property asset of the software vendors, and how they work is protected as a trade secret and competitive advantage. But knowing how the algorithms that shape our users experience of our collections and services is essential if we are to make informed decisions around software licensing and development, user and instructional support, and collection development. I've been experimenting with methods for auditing algorithms by assessing large results sets to determine patterns and screen for systemic problems and biases. In this presentation, I'll discuss the methods I've used for algorithmic audits, the potential impacts of algorithmic auditing on library operations, and auditing algorithms without violating the software's Terms of Service."
