Together AI Compiler Makes AI Responses Up to 2.6x Faster
Together AI has released Together Compiler, a tool where AI agents keep tuning how models run as user traffic changes. In tests it beat the best-known configurations by up to 2.6 times on speed, and each model was optimised in under 24 hours.
Quick glossaryTogether AI, inference, InferenceX
- Together AI
- An AI cloud company best known for running and hosting open-source AI models for developers.
- inference
- The step where a trained AI model takes your input and produces its answer, which can be slow or costly.
- InferenceX
- A benchmark that compares how well different setups run AI models, used here as the yardstick.
Burning Potato score: 64%