- throughput:
- 28.0 t/s gen
Prefill performance mentioned but no numbers given. Decode speeds at various depths: 28 t/s at start, 23.5 t/s at 45k, 18 t/s at 192k. Maintained with 8k token output.
M2 Ultra 192GB
APPLE · 192GB unified memory · 1 report
This page is thin (1 of 3 reports needed for indexing). Help fill it in.