“Cite me, LLM!”: How to get your evidence noticed (and quoted correctly) in the age of AI

With the rise of large language models (LLMs) and other generative AI systems, an ever-increasing proportion of the information in the world around us is either generated by AI, or is searched for and interpreted by AI systems before being presented to users.

The rapidly increasing role for AI in how information is generated, searched for and curated has profound implications for how medical and scientific evidence is communicated.  

For example, it is very likely that at this moment, a physician in a busy health clinic is about to see a patient with an unfamiliar diagnosis and asking ChatGPT for information to help them quickly get up to speed on the topic. Likewise, many people use online information to help them self-diagnose or manage health conditions, and will increasingly be presented with AI-generated summaries when they search for health topics online. After a twenty-year period where internet searches were king, AI systems are increasingly shaping how we find information and learn new knowledge.  

For medical and scientific communications, this raises some specific challenges. Many AI systems now have the ability to search for information online, but getting noticed in these searches may be challenging and it may not be obvious the search strategies that AI systems use. Likewise, many AI systems generate summaries or reports for users, but how do the AIs prioritise what information to include and cite in these summaries? AI systems also make mistakes and might hallucinate plausible looking but incorrect information, running the risk that medical or scientific findings are interpreted or cited incorrectly.  

This is an incredibly fast-moving space, with new AI capabilities emerging almost every week. Here are our current top tips for navigating this new world of medical communications: 

Publish outside paywalls 

LLMs are largely trained on information which is openly available on the internet, and many state of the art AI systems use internet searches to find information when completing a task such as creating a research report. It is reasonable to assume that anything behind a paywall is essentially invisible to LLMs – neither incorporated into the knowledge embedded in the model itself, nor available for AI systems to find and use through a search. To overcome this, aim to publish full text publications on the web under open access licences, or consider publishing pre-prints on sites such as medRxiv.org. Where feasible, publishing in native web formats rather than PDFs will make it easier for AI systems to find and use your publication. 

Double down on crafting excellent abstracts 

The abstract has always been the most important section of research publications for impact and accurate citations. Many readers - and now, many LLMs - will only ever engage with the abstract. Its content often determines whether the full text publication is included in literature reviews and meta-analyses, or AI-generated summaries. It is worth scrutinising and crafting every word and data point in an abstract to make sure that it maximises the communication value and accuracy of the abstract. The conclusion section is often the most challenging to make really excellent, but it is likely that what you write here will heavily influence how LLMs will interpret and summarise the research. 

Create LLM optimised supplementary content 

Supplementary communications such as visual abstracts, blogs, posters, videos and whitepapers can help to communicate medical and scientific findings to different types of audience and to amplify reach on social media.  

LLMs work best when content is clean, structured, and publicly accessible. Including relevant keywords and hashtags (e.g. “#cardiology”) will help LLMs find the work and understand the main themes of the publication.   

There are a few writing and design tactics you can use when creating supplementary content to increase the chance of being noticed and cited correctly by LLMs:

• Use a clear logical structure, with headings, subheadings, and bullet points to summarise key messages.
• Break the content into digestible chunks.
• Use visuals, infographics, charts and tables to add clarity or context.
• Use question-based headings (e.g. “What does this mean for patients?”).
• Clearly cite authoritative sources, and avoid using DOIs here (LLMs love to hallucinate these!).

Wrap up

It is early days for AI optimised medical communications, but this is evolving very quickly and the direction of travel is clear. The robots have learnt how to read, and it’s time we learnt how to write for them.

At LCP Health Analytics we are working at the cutting edge of AI, supporting clients in the life sciences and healthcare industries to use these new technologies to achieve success in evidence generation, HEOR and market access. Please drop me an email if you’d like to connect and discuss how we can help (ben.bray@lcp.uk.com).