llama lora chinese couplet

llama-lora-chinese-couplet

A simple llama-lora end to end example to show its potential to train a Chinese Couplet AI

• int8 LLaMa + LoRA = Finetune in a consumer GPU <= 10G vram!

Slides, last update 04/29/2023

• LLM, Pretrain, Finetune
• LoRA: A popular Parameter Efficient Finetune approach by Microsoft
• LLaMA: A gift from Facebook to the research community
• LLaMA-2: A newer and better gift from Facebook
• Alpaca: A $600 approach to “distill 80%” of ChatGPT by Stanford
• Alpaca-LoRA: An even cheaper approach from a Stanford student?
• Omitted due to time
  • Quantization intro, LLM.int8

Zero-shot Examples

• After 3 epochs of 5k pairs, cap max tokens, greedy
• post-processing to match # of chinese chars (so yes, I cheated ^_^)
• Ideally a well trained model will know end of sentence (eos) itself
• prompt:对联：{上联}n下联：

Top link	Base LLaMA	LLaMa_LoRA_A100_9mins	LLaMa_LoRA_Tesla_T4_35mins
The spring breeze is proud to pave the road	Immerse yourself in the mudnPrevious link	Moonlight listens to the sound of the wind blows dreams	The wind and rain blow the dreams floating
Beautiful China Charming Beijing	Beautiful China Charming BeijingnPrevious link:	The history of China's dream	Dream China Dream Acer
The dream of a fish book	Fish roasted porkn	Birds sound thousands of miles	Birds sound thousands of miles
Sunset and sunset to the ancient temple	Sunset shines on the ancient templen	The moon reflects the morning rain fills the dream	The moon reflects the morning frost fills the dream

• In case you are into Chinese couplets, I have a better T5 version

Some additional thoughts on high-quality AI

• See the notes