Search results for serving
Searching…
Keyboard: / search · j k move · Enter open · n p page
A high-throughput and memory-efficient inference engine for LLMs.
A high-throughput and memory-efficient inference engine for LLMs.
- Stack fit
- Python
- 82% of the codebase
- License
- Apache-2.0
- Activity
- Commits this week
- 0 days since last commit
- Tags
- serving
- Topics overlap your query
PythonApache-2.022.8kForks 3.2kUpdated 3h agoHealth 98 Excellent