Five days after the first GPUShare post, I split the original FastAPI application into a public middleware layer and a hardware-facing backend. That boundary remains the most important architectural change, but the current code also makes some of the original claims more precise.
The backend split
Public middleware (port 8000) handles JWT and API-key authentication, users and invites, account aggregation, append-only ledger views, optional Stripe integration, MCP configuration, and short-lived in-process caches. This is the service the Cloudflare Tunnel targets.
Hardware backend (port 8080) handles Ollama and optional OpenRouter requests, Blender submission and rendering, R2 file operations, GPU and integration health checks, and MCP tool execution. Middleware calls it with an X-Internal-Token shared secret and an asserted user ID.
The backend is not a Cloudflare ingress target, but Docker Compose currently publishes port 8080 on the host. Its shared-secret check is therefore an application security boundary, not proof of complete network isolation. The backend also makes outbound calls when OpenRouter, R2, or other integrations are enabled.
Inference streams are proxied as Server-Sent Events. Model and health responses use short-lived process-local caches, and the account page uses one aggregate endpoint instead of a fan-out of browser requests.
OpenAI compatibility and routing
The OpenAI-compatible endpoints support streamed and non-streamed chat, model discovery, image-capable content shapes, and structured tool calls. Ollama executes locally through a FIFO queue. OpenRouter is an optional route for selected cloud models, while auto chooses from configured local or cloud options using token thresholds and user preferences.
This is routing, not failover: a failed local request is not automatically retried against OpenRouter. That distinction matters for availability, cost, and prompt privacy.
Usage accounting
The account totals now sum explicit ledger types for local inference, cloud inference, and rendering instead of deriving chart segments from limited recent logs or unrelated balance movements. The ledger is append-only and balances are calculated from its rows.
Local charges are still estimates. They combine configured or detected wattage, the electricity tariff, estimated token counts and throughput, or measured render duration. NVIDIA detection reads the configured power limit as an input assumption; it does not measure each request’s wall-power draw. Optional Tapo monitoring reports whole-device telemetry, not request-level attribution.
Blender rendering and its trust model
The render path validates .blend uploads, removes Blender text blocks, stores files in optional R2, queues a separate worker, records frame progress, packages output, and writes a render-usage ledger entry. Text-block removal reduces one risk but is not a full sandbox, so this remains a trusted-user workflow.
Setup automation—and the current regression
The macOS/Linux and Windows scripts detect hardware, recommend model sizes, estimate wattage, generate secrets, configure optional services, build containers, and check health. However, their database migration and troubleshooting commands still refer to the old fastapi Compose service. The current Compose file names the services middleware, backend, and render-worker, so that step needs repair before “one-click install” is accurate again.
Current status
The implemented product includes account and admin workflows, API keys, local and cloud model routes, streaming and tool calls, MCP server management, queued Blender jobs, and detailed usage accounting. It is still host-dependent software for a trusted group—not a managed, high-availability, sandboxed, or production-hardened cloud service. No adoption claim is implied.
The code is open source at github.com/Slaymish/GPUShare.