Appearance
Gaps & Roadmap Improvements
- P0: Implement speculative decoding to accelerate decode iterations.
- P1: Add tensor-parallel and pipeline-parallel multi-GPU partitioners.
- P2: Implement host-to-device KV cache swapping for memory oversubscription.