TensorSharp supports Vulkan backend

Short summary

TensorSharp released initial Vulkan backend support, enabling GPU acceleration on Nvidia and Intel hardware. Benchmark comparisons with llama.cpp show competitive-to-superior throughput on decode, prefill, and time-to-first-token. The open-source LLM inference engine supports Unsloth models, multi-modal features, and OpenAI-compatible API; AMD GPU testing requested.

• •Vulkan backend now available for GPU acceleration on Nvidia and Intel GPUs
• •Benchmarks show competitive or superior performance vs llama.cpp across multiple metrics
• •Open-source engine with multi-modal support and OpenAI-compatible API; needs AMD testing

Generated with AI, which can make mistakes.