i14 trains and serves Laplace 3B linear attention model on AWS Trainium2
Fixed-size state
Serving memory stays fixed as context grows, avoiding KV cache growth on Trainium2
i14, working with AWS's GenAI Innovation Center, rebuilt its Laplace 3B linear attention model for AWS Trainium2 chips using native PyTorch via TorchNeuron Native. The design uses a fixed-size recurrent state, so serving memory stays flat as context length grows.
Read the full story at AWS Machine Learning
The tools behind this win
- AWS Trainium2Hardware used to train and serve the Laplace 3B model
- TorchNeuron NativeNative PyTorch path used to port the model to Trainium2