• 0 Posts
  • 90 Comments
Joined 3 months ago
cake
Cake day: March 5th, 2025

help-circle




  • I’m trying to understand how this is used, my current selfhosted setup is quite simple:

    frontend - backend - vllm

    Using the openai compatible api vllm returns with tool calls that need to be handeled by the backend, an results are then send to the frontend by the backend, the frontend never communicates with vllm directly.

    Form what i can gather AG-UI is a way for the frontend to directly communicate with in my case vllm in a secure way? letting the frontend handle the tool calls?