How It Works: Retrieval-Augmented Generation Flow for OneSearch AI Research Assistant

Flowchart showing how a user’s research question is processed in two stages: Retrieval and Generation. In the Retrieval stage, the query searches Clarivate sources like Web of Science and ProQuest. Results with abstracts and metadata are combined with the original query. In the Generation stage, the combined prompt is sent to OpenAI GPT 4/4o mini, which returns summaries, answers, references, and related research questions.

The Research Assistant uses a Retrieval Augmented Generation (RAG) architecture to combine the language capabilities of the LLM (Large Language Model; currently GPT 4.1-mini) with the knowledge from material in the Central Discovery Index (an index of scholarly and academic electronic content that includes the library's subscriptions).

The tool converts a user's natural language question into a Boolean query using a large language model (LLM). This query is then sent to the Central Discovery Index (CDI) to retrieve results. The top results (up to 30) are then analyzed, and the five most relevant sources are identified based on their abstracts and metadata. The LLM then creates a concise summary with inline references based on those top five results, which is then delivered to the user.

In a nutshell:

  • 1. Query Conversion:The user's question is sent to the LLM, where it is converted to a Boolean query that contains a number of variations of the query, connected with an OR. If the query is non-English, some of the variations will be in the query language, and the other variations will be in English.
  • 2. Results Retrieval:The Boolean query is sent to CDI to retrieve the results.
  • 3. Re-ranking:The top results (up to 30) are re-ranked using embeddings to identify five sources that best address the user's query.
  • 4. Overview Creation:The top five results are sent to the LLM with the instructions to create the overview with inline references, based on the abstracts.
  • 5. Response Delivery:The overview and sources are returned to the user in the response.