TL;DR# I find FlashAttention easiest to understand through one question: what state must survive after each score or tile is processed?
The key idea is that an intermediate output can change meaning, as long as the final …
TL;DR# DFlash2 + Qwen3-27B + OpenCode made a local coding-agent workflow practical for me: it completed my MNIST task within half an hour. These are observations from my experiment write-up.
The prompt was simple: write …
DeepSeek has created a huge wave of discussion and panic in the market.
I think it’s great overall.
Innovation From a technology standpoint, DeepSeek is truly pushing the envelope in both training methods and model …
TL;DR
Nvidia’s DIGITS offers an on-premise AI solution aimed at smaller organizations that require strict data privacy. While it is cost-effective for small businesses and college research labs, it may be less suitable …
#software #accelerator #ai #on-device #WIP
In this part of the tutorial, we will learn how to run a pretrained model from TorchVision: ResNeXt50, which is a model architecture built upon the concepts of ResNet. …
#software #ai #llm #open-source
How to train a transformer based model, like LLaMA2, from scratch? Andrej Karpathy has open-sourced llama2.c project on GitHub. My learning process is “duplicate and rewrite”, …
#software #accelerator #ai #on-device
SNPE = Snapdragon Neural Processing Engine
In this tutorial we assume that Qualcomm SNPE has been successfully installed use QPM. Follow “Qualcomm Package Manager 1.0” …
Here is my notes from JPMorgan’s “Eye on Market” 2024 April issue
The emergence and integration of large language models (LLMs) into various professional sectors have marked a significant milestone in …
Let’s consider a face recognition system, where we’ve got facial images from a list of known persons and the system input is a camera image. We need to figure out if there are people in this camera image from …
LLM is a memory bound problem. This inspired me to look at different memory technologies. In this article, I’m going to summarize my research these days, especially about HBM and its impact on AI applications. …
In this Github project, I created a simple application that can do text to image and image to image search, using open-source transformer model. Details can be found in the repo and its docs directory.
Here is a screen …
#software #ai #llm #open-source
I was involved in the early ExecuTorch definition phase and had used its predecessor Lite Interpreter extensively in work. I really like this idea and its design. This is a great effort …
To optimize the efficiency of training or executing an ML model, whether implemented locally on a device or hosted in the cloud, parallelization plays a critical role, akin to other computational challenges.
Utilizing …
#software #ai #llm #open-source
https://github.com/jimwang99/understanding-llama2/tree/main/pytorch
Above GitHub repo is an implementation of LLaMA2 and test-case use TinyStories in PyTorch
Example output
------ …
#software #ai #llm #open-source
Following up with Understanding LLaMA2 Part 1 Model Architecture, this diagram explains LLaMA model architecture with KV Cache support. We follow the same legend as well as the …
#software #ai #llm #open-source
Here I’m capturing the details of llama’s model architecture use PlantUML component diagram, using the following 2 GitHub repos as references …