Understanding LLaMA2

Browse the Understanding LLaMA2 series.

Posts without dates

Understanding LLaMA2 Part 5 Training with TinyStories

#software #ai #llm #open-source How to train a transformer based model, like LLaMA2, from scratch? Andrej Karpathy has open-sourced llama2.c project on GitHub. My learning process is “duplicate and rewrite”, …

Understanding LLaMA2 Part 4 ExecuTorch Runtime

#software #ai #llm #open-source I was involved in the early ExecuTorch definition phase and had used its predecessor Lite Interpreter extensively in work. I really like this idea and its design. This is a great effort …

Understanding LLaMA2 Part 3 PyTorch Implementation

#software #ai #llm #open-source https://github.com/jimwang99/understanding-llama2/tree/main/pytorch Above GitHub repo is an implementation of LLaMA2 and test-case use TinyStories in PyTorch Example output ------ …

Understanding LLaMA2 Part 2 KV Cache

#software #ai #llm #open-source Following up with Understanding LLaMA2 Part 1 Model Architecture, this diagram explains LLaMA model architecture with KV Cache support. We follow the same legend as well as the …
Backward Machine Learning Understanding LLaMA2 Part 1 Model Architecture Forward