Having an assessment tool is an interesting idea. I have read/seen some articles on utilizing AI in library assement but have not seen clear examples of assessing the AI itself from a library perspective. I am also curious if someone has already developed an AI evaluation tool they want to share.

We currently utilize standard resource workflows from staff/faculty reviews/feedback to gadge the usefulness of a specific AI tool and if we would like to consider it for adoption. We don’t utilize a formulaic approach, at least not now.

Davin Pate, M.L.S.

Assistant Director for Scholarly Communications and Collections

(972) 883-2908 |davin.pate@utdallas.edu

http://www.utdallas.edu/library/

The University of Texas at Dallas

Book time to meet with me

From: Bunton, Glenn <BUNTONGA@mailbox.sc.edu>
Sent: Tuesday, April 1, 2025 2:26 PM
To: ai-sig@exlibrisusers.org
Subject: [Ai-sig] Re: Topic Discussion (Ex Libris/ProQuest/Clarivate AI Products)

We are currently trying to develop an evaluation/assessment tool for use with AI/GenAI research products. I suspect many others have similar interests and may be working on related projects. Our goals/purposes are primarily to try and assess which tool(s) might be worth pursuing a paid subscription relative to others. If anyone has already developed something that might be useful for that purpose, would you be willing to share? If you know of rubrics or assessment tools that might be useful, that would be great as well.

Thanks in advance for any information and suggestions.

__

Glenn Bunton
Data Visualization Librarian

803-777-2903
buntonga@mailbox.sc.edu

Digital Research Services
University Libraries
University of South Carolina
1322 Greene Street
Columbia, SC 29208

From: Zhang, Hui <Hui.Zhang@oregonstate.edu>
Sent: Tuesday, April 1, 2025 3:10 PM
To: Pate, Davin <djp130330@utdallas.edu>; ai-sig@exlibrisusers.org <ai-sig@exlibrisusers.org>
Subject: [Ai-sig] Re: Topic Discussion (Ex Libris/ProQuest/Clarivate AI Products)

Hello, all:

Thank you Davin for initiating this conversation! We at Oregon State University are investigating several AI tools, including Primo Research Assistant, Web of Science RA, and other vendors (Elicit, Scite). For Primo RA, we have:

Enabled it in a test view
Offered a public webinar on AI tools for libraries

Initial feedback we have received:

What are the use cases for Primo RA? To patrons, it is unclear what are the benefits and advantages of using it over other AI tools or Primo.
IZ and NZ records are unavailable in Primo RA.
What patrons like is the multilingual feature. For example, you can search in Chinese, get English articles in the results, and the summary is in Chinese. That feature is missing in Primo, but is it available in NDE?

I hope to track the information on a spreadsheet and provide it to the related working groups/steering.

Maybe we can discuss the strategy as a group. ELUNA? I am planning to attend it, but only on the 19^th and 20^th.

Thanks

Hui

From: Pate, Davin <djp130330@utdallas.edu>
Date: Monday, March 31, 2025 at 2:30 PM
To: ai-sig@exlibrisusers.org <ai-sig@exlibrisusers.org>
Subject: [Ai-sig] Topic Discussion (Ex Libris/ProQuest/Clarivate AI Products)

You don't often get email from djp130330@utdallas.edu. Learn why this is important

[This email originated from outside of OSU. Use caution with links and attachments.]

Hi Everyone,

I wanted to begin our initial Topic Discussion series with the following:

What Clarivate/ProQuest/Ex Libris AI Products/Tools are you or your university currently investigating? What have you liked about the products and have you seen any Initial problems or concerns regarding the products?

If you have seen a significant concern, what case have you submitted, if any, and the case number associated with the issue.