vLLM — Developer tools for optimizing and serving AI model inference.
Efficient model serving and inference and the PyTorch Foundation page lists vLLM among its projects.
- Recorded website languages
- English
Official sources: vllm.ai
Related links
SGLang
Developer tools for optimizing and serving AI model inference.
DeepSpeed™
Tools for distributed AI training, inference and workload scaling.
Transformers (Hugging Face)
Machine-learning framework, developer documentation and model training tools.
Safetensors
Model formats, interoperability and tools for loading model data.
Replit
Online development platform for creating, running and deploying software projects.
Ollama
Tools for running and managing language models.
NVIDIA NeMo
Machine-learning framework, developer documentation and model training tools.
Streamlit
Python tools for building and sharing data and AI applications.