Hi Mark, I've come across various pieces of information that might answer (or begin to answer) your questions using the Primo Research Assistant as another example. The getting started<https://knowledge.exlibrisgroup.com/Primo/Product_Documentation/020Primo_VE/...> documentation talks about using RAG architecture<https://aws.amazon.com/what-is/retrieval-augmented-generation/> to reduce risks of hallucinations and the AI privacy policy<https://clarivate.com/ai/academia/policy/> linked mentions that "Clarivate does not use your data to directly or indirectly train LLMs." So, take that how you will! They also mention using OpenAI's GPT--4o mini for the current AI model. Clarivate hasn't mentioned a specific threshold for excessive use for the PRA, but they do say "Currently, there are no restrictions on the number of searches. However, we are monitoring usage to control costs for the Large Language Model and may need to set a limit on the number of searches an institution can perform within a certain timeframe in the future." Best, Alison ////////////////////////////////////////////// Alison L. Davis Discovery and Integrated Library Systems Administrator Purdue University Libraries and School of Information Studies 504 Mitch Daniels Blvd | West Lafayette, IN 47907 lampley@purdue.edu<mailto:lampley@purdue.edu> | Schedule a Teams meeting with me!<https://calendar.lib.purdue.edu/appointments/lampley/teams> From: Erin Nettifee (she/her/hers) <erin.nettifee@duke.edu> Sent: Thursday, April 10, 2025 10:30 AM To: Kluzek, Mark <m.kluzek@ucl.ac.uk>; ai-sig@exlibrisusers.org Subject: [Ai-sig] Re: Ex Libris & AI - technical documentation You don't often get email from erin.nettifee@duke.edu<mailto:erin.nettifee@duke.edu>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> ---- External Email: Use caution with attachments, links, or sharing data ---- Hi Mark, Using the Metadata Assistant as an example, they are not getting into technical details, but they are giving some more in-depth responses in the FAQ: https://knowledge.exlibrisgroup.com/Alma/Product_Documentation/010Alma_Onlin... "Q: What LLM are you working with? A: Models are subject to change as models and capabilities evolve. As of the February 2025 release, the AI Metadata Assistant was working with the gpt_4o_2024_08_06 LLM model." Best, Erin Nettifee Duke University Libraries ________________________________ From: Kluzek, Mark Sent: Thursday, April 10, 2025 10:18 AM To: ai-sig@exlibrisusers.org<mailto:ai-sig@exlibrisusers.org> Subject: [Ai-sig] Ex Libris & AI - technical documentation Hi all, I've been wondering whether there are any technical documents that outline, in some depth, how AI works with Ex Libris products. I'm thinking along the lines of the documentation you see in the Developer Network. Specifically, I was hoping it might address questions such as: * What specific AI models are used for the different AI-related functionalities in Alma? Architectural diagrams outlining the flow of data would be useful. * Is there a usage threshold in the event of excessive use? * What risks are there of AI "hallucinations"? How do the models ensure the greatest possible accuracy of information? * In terms of privacy, is the data we input retained or used in any way? I've engaged a little with Ex Libris, as well as with other members of the Ex Libris user community. I don't believe anything like this currently exists, but I do think there would be value in greater transparency around how AI is being used. I'd be interested to hear if anyone else has similar thoughts or insights on the matter. Best regards, Mark

Hi Everyone, I received this from our Steering Liaison, Ryan Edwards. I am watching it now, and it could help answer a few of the questions below. Thanks, Ryan, for sending it to me. Evaluating the quality of generative AI output - Methods, metrics and best practices<https://videos.clarivate.com/watch/hB3gJyznrpbZ7uKhGLoQjp?utm_campaign=EM3_A...> Davin Pate, M.L.S. Assistant Director for Scholarly Communications and Collections (972) 883-2908 |davin.pate@utdallas.edu<mailto:%7Cdavin.pate@utdallas.edu> http://www.utdallas.edu/library/ The University of Texas at Dallas [cid:image001.png@01DBAA1D.79DCAB40]<https://outlook.office.com/bookwithme/user/af697753dcc24b9e926afe35e8e84e9d@...> Book time to meet with me<https://outlook.office.com/bookwithme/user/af697753dcc24b9e926afe35e8e84e9d@...> From: Pate, Davin <djp130330@utdallas.edu> Sent: Thursday, April 10, 2025 10:15 AM To: Kluzek, Mark <m.kluzek@ucl.ac.uk>; ai-sig@exlibrisusers.org Subject: [Ai-sig] Re: Ex Libris & AI - technical documentation Hi Mark, I concur with several questions and the need for greater documentation and transparency in Ex Libris/Clarivates approach. I forwarded this to a few people I know at Ex Libris who are involved with AI/AI products to think about and comment on. In addition to this, I would encourage everyone to take a look at the AI and emerging technologies track at ELUNA this year https://elunaconf2025.dryfta.com/program-schedule/program/pstrack/2/ai-emerg.... I outlined the sessions that could lead to some interactions with Ex Libris/Clarivate, which could provide additional opportunities to bring up your questions. The next wave of GenAI: AI Agents and beyond at Clarivate<https://elunaconf2025.dryfta.com/program-schedule/program/detail/159/the-nex...> AI in the loop<https://elunaconf2025.dryfta.com/program-schedule/program/detail/24/ai-in-th...> Beta-testing the Alma AI Metadata Assistant<https://elunaconf2025.dryfta.com/program-schedule/program/detail/51/beta-tes...> Keeping Libraries at the Forefront of AI in the Classroom<https://elunaconf2025.dryfta.com/program-schedule/program/detail/11/keeping-...> Davin Pate, M.L.S. Assistant Director for Scholarly Communications and Collections (972) 883-2908 |davin.pate@utdallas.edu<mailto:%7Cdavin.pate@utdallas.edu> http://www.utdallas.edu/library/ The University of Texas at Dallas [cid:image001.png@01DBAA1D.79DCAB40]<https://outlook.office.com/bookwithme/user/af697753dcc24b9e926afe35e8e84e9d@...> Book time to meet with me<https://outlook.office.com/bookwithme/user/af697753dcc24b9e926afe35e8e84e9d@...> From: Kluzek, Mark <m.kluzek@ucl.ac.uk<mailto:m.kluzek@ucl.ac.uk>> Sent: Thursday, April 10, 2025 9:18 AM To: ai-sig@exlibrisusers.org<mailto:ai-sig@exlibrisusers.org> Subject: [Ai-sig] Ex Libris & AI - technical documentation Hi all, I've been wondering whether there are any technical documents that outline, in some depth, how AI works with Ex Libris products. I'm thinking along the lines of the documentation you see in the Developer Network. Specifically, I was hoping it might address questions such as: * What specific AI models are used for the different AI-related functionalities in Alma? Architectural diagrams outlining the flow of data would be useful. * Is there a usage threshold in the event of excessive use? * What risks are there of AI "hallucinations"? How do the models ensure the greatest possible accuracy of information? * In terms of privacy, is the data we input retained or used in any way? I've engaged a little with Ex Libris, as well as with other members of the Ex Libris user community. I don't believe anything like this currently exists, but I do think there would be value in greater transparency around how AI is being used. I'd be interested to hear if anyone else has similar thoughts or insights on the matter. Best regards, Mark

Hi Lili, * Thanks for sharing this. This does address a number of the points I raised, which is great. Is there any chance of something similar being published for the Primo Research Assistant? Also, is there an outline anywhere regarding how the data we input is retained/used? Best regards, Mark ________________________________ From: Lili Daie <Lili.Daie@Clarivate.com> Sent: 14 April 2025 12:34 PM To: Kluzek, Mark <m.kluzek@ucl.ac.uk>; ai-sig@exlibrisusers.org <ai-sig@exlibrisusers.org> Subject: RE: Ex Libris & AI - technical documentation ⚠ Caution: External sender Hi Mark, Thank you for raising this! I agree that it is important to have clear information on how AI is being used in AI features. Accordingly, when documenting the Alma AI Metadata Assistant we aimed for transparency and clarity on how AI is being used, including an FAQ section with answers to questions we’ve often received from libraries (such as the LLM used, whether uploaded images are saved or used in any way, etc.) - to make sure they are clearly addressed in the documentation. You can find that information here: https://knowledge.exlibrisgroup.com/Alma/Product_Documentation/010Alma_Onlin... Regards, Lili Lili Daie | Manager, Product Management Pronouns: She/her/hers Time Zone: IST/IDT [cid:image001.png@01DBAD4A.53EF2BD0] [A black letter f in a grey square Description automatically generated]<https://www.facebook.com/exlibrisgroup> [A black x in a grey square Description automatically generated] <https://twitter.com/ExLibrisGroup> [A black and grey square with a letter in it Description automatically generated] <https://www.linkedin.com/company/ex-libris/> [A black and grey square with a play button Description automatically generated] <https://www.youtube.com/user/ExLibrisLtd> From: Kluzek, Mark <m.kluzek@ucl.ac.uk> Sent: Thursday, April 10, 2025 5:18 PM To: ai-sig@exlibrisusers.org Subject: [Ai-sig] Ex Libris & AI - technical documentation Hi all, I've been wondering whether there are any technical documents that outline, in some depth, how AI works with Ex Libris products. I'm thinking along the lines of the documentation you see in the Developer Network. Specifically, I was hoping it might address questions such as: * What specific AI models are used for the different AI-related functionalities in Alma? Architectural diagrams outlining the flow of data would be useful. * Is there a usage threshold in the event of excessive use? * What risks are there of AI "hallucinations"? How do the models ensure the greatest possible accuracy of information? * In terms of privacy, is the data we input retained or used in any way? I've engaged a little with Ex Libris, as well as with other members of the Ex Libris user community. I don't believe anything like this currently exists, but I do think there would be value in greater transparency around how AI is being used. I'd be interested to hear if anyone else has similar thoughts or insights on the matter. Best regards, Mark Confidentiality note: This e-mail may contain confidential information from Clarivate. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this e-mail is strictly prohibited. If you have received this e-mail in error, please delete this e-mail and notify the sender as soon as possible.