I've seen some domain applications where a BERT model of 10Million parameters with knowledge about a domain (over an LLM with 1 billion parameters) is good enough. This is a BERT model already trained on Material Science: https://huggingface.co/m3rg-iitd/matscibert If the researcher is set on utilizing an LLM, it might be easier to simply fine-tune a pre-trained model on the MatScie dataset https://github.com/M3RG-IITD/MatSciBERT LoRA Fine tuning : https://www.databricks.com/blog/efficient-fine-tuning-lora-guide-llmsReceived on Sat Apr 27 2024 - 10:11:20 EDT