To fine-tune or not to fine-tune

When adopting general models like GPT-4 and LLaMA-2 to specific areas, such as healthcare, we can use different strategies, including prompt engineering, parameter-efficient fine-tuning, instruction tuning, and re-training from scratch (ordered by technical difficulty to implement, figure 1). People usually believe that fine-tuning with domain-specific knowledge and data can yield better results than prompt engineering. Is it necessary to do fine-tuning, which can cost significant computing resources but probably gain only marginal performance improvement?

In a recent manuscript published by Microsoft, the authors showed that GPT-4 with a composition of several prompting strategies (named Medprompt, figure 2) can achieve superior performance in multiple medical challenge benchmarks in the absence of fine-tuning. These strategies include (figure 3):

1. Few-shot prompting: Provide a few examples of correct Q&A. The study developed a kNN-based few-shot, which selects the examples based on K-nearest text embeddings from previous correct Q&A.

2. Chain-of-thought: Guide the LLM to think step by step

3. Ensembling: Produce multiple outputs and then consolidate to a consensus output

Another study also found that Flan-T5 with minimal prompt engineering still achieves high fidelity when analyzing clinical notes.

A busy week for AI-related business deals

• Boehringer plugs in IBM-trained AI model to boost antibody drug discovery efforts (source)

• Boehringer (again) bets $509M on Phenomic’s tumor-targeting tech, as AI biotech plans for busy year (source)

• Phenomic (again) enters into strategic research collaboration with Astellas for solid tumor cell therapies (source)

• Johnson & Johnson bets heavy on AI-driven drug discovery (source)

• Amgen expands pact with Amazon to usher drug manufacturing into the AI era (source)

• AstraZeneca will pay Absci up to $247 million to collaborate on the design of an anti-cancer antibody using AI (source).

Other important papers to read

• An ethical principle, namely “GREAT PLEA” (Governability, Reliability, Equity, Accountability, Traceability, Privacy, Lawfulness, Empathy, and Autonomy), is proposed for generative AI in healthcare. A Nature comment also recommends that medical professionals, not commercial interests, must drive their development and deployment to protect people’s privacy and safety.

• We previously mentioned the need for multimodal AI models for healthcare, and here comes one from the University of Oxford, called Med-MLLM, which supports both image and text data.

##########

If you find the newsletter helpful, please consider:

• 🔊 Sharing the newsletter with other people

• 👍 Upvoting on Product Hunt

• 📧 Sending any feedback, suggestions, and questions by directly replying to this email or writing reviews on Product Hunt

• 🙏 Supporting us with a cup of coffee.

Thanks, and see you next time!