Spaces:

ResearchEngineering
/

AGI

Running

App Files Files Community

163 kB

Ctrl+K

Ctrl+K

2 contributors

History: 79 commits

Dmitry Beresnev

Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling

27c1c3c 16 days ago

cpp
Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling 16 days ago
python
Refactor the C++ LLM manager into modular components, moves Python modules under python/, and keeps the current control-plane behavior intact. The C++ server now has clearer separation for config, model lifecycle, runtime services, request parsing, HTTP helpers, and server routing, while Docker build/runtime paths were updated to compile multiple C++ files and load Python code from the new package folder 4 months ago
.clang-format

69 Bytes
add cpp server 5 months ago
.gitattributes

1.52 kB
initial commit 8 months ago
.gitignore

1.21 kB
add auth, token policy, queue scheduler, and cancel flow, etc 4 months ago
Dockerfile

6.17 kB
Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling 16 days ago
README.md

11.5 kB
fix description 4 months ago
config.toml.example

1.17 kB
Add inference metrics + bench script, tune threads/batch for 2-vCPU HF Space, backoff on readiness polling 16 days ago
pyproject.toml

471 Bytes
fix gitignore, app and logger, etc 6 months ago