Work Article

I'm Building a GPU PC, So I Made It a Private AI Server for My Friends

· 4 min read
self-hostedAIollamacloudflare
Project GPUShare
Live Demo GitHub

GPUShare started with a practical question: if a desktop GPU is idle most of the day, can a trusted group share it without turning a home PC into a pretend cloud region?

The current project is an open-source, self-hosted application for local AI inference and queued Blender rendering. It includes password and invite-based accounts, API keys, account and admin views, an OpenAI-compatible chat API, optional cloud-model routing, MCP tool connections, usage accounting, and a public React interface.

The current request path

Browser or API client
        ↓
Vercel-hosted React frontend
        ↓
Cloudflare Tunnel → public FastAPI middleware
        ↓  X-Internal-Token
private hardware backend
        ├─ Ollama on the host
        ├─ Blender render worker
        ├─ optional Cloudflare R2 file storage
        └─ optional OpenRouter cloud models

The middleware owns JWT and API-key authentication, account and admin operations, billing integration, aggregation, and short-lived in-process caches. Hardware requests are forwarded to a second FastAPI service using a shared secret. The Cloudflare ingress points at the middleware, not the backend. Docker Compose does still publish the backend port on the host, so the application-level internal token remains an important boundary.

Local AI, with an explicit cloud option

The standard /v1/chat/completions and model endpoints follow OpenAI-compatible shapes. Responses can stream over Server-Sent Events, and structured tool calls are normalised for both Ollama and OpenRouter. Local Ollama requests are serialised through a process-local GPU queue.

OpenRouter is optional. A selected OpenRouter model—or the configured auto model when its selection rules choose cloud—sends the request to that provider. It is not an automatic failover that retries a failed Ollama request. When local-only routing is selected, model inference runs on the host; when a cloud route is selected, prompt data leaves the host under that provider’s terms.

What “electricity cost” means in the implementation

GPUShare does not directly meter the wall-power consumed by each inference request. It estimates local usage with configured or detected GPU and system wattage, the configured electricity tariff, estimated token counts, and a default throughput assumption. Render estimates use elapsed job time and configured wattage. Optional OpenRouter requests use provider pricing.

estimated local cost = estimated kWh × configured electricity tariff

The calculator below is illustrative and uses the same kind of assumptions; it is not a utility-meter reading.

GPUShare Cost Calculator

See what local inference actually costs on a 5070 Ti vs cloud APIs.

504,000
1500
$0.01$0.60
Per response
NZ$0.0001
Daily
NZ$0.0066
Monthly
NZ$0.20
vs cloud
2x cheaper
Cloud equivalent (GPT-4o mini): $0.45/mo
Local (Llama 3.1 8B): NZ$0.20/mo

Local cost = 150W GPU draw x inference time x electricity rate. Cloud comparison uses GPT-4o mini output token pricing. Actual throughput varies with quantisation, context length, and batch size.

The accounting record itself is append-only: usage charges, cloud usage, render usage, top-ups, payments, and adjustments are separate ledger entry types, and the displayed balance is their sum.

Blender rendering is a trusted-user feature

Users can submit .blend files with frame, resolution, engine, sample, and output settings. The backend validates the file type and size, removes Blender text blocks, uploads the cleaned file to optional R2 storage, and queues a separate worker. Completed output is packaged and exposed through a time-limited signed URL.

Removing embedded text blocks is a useful guardrail, but it is not a full process, filesystem, network, or resource sandbox. The current upload path is suitable only for trusted invited users, not arbitrary public files.

Operational limits

  • If the host or Cloudflare Tunnel is offline, local inference and render submission are offline.
  • Cloud routing is optional model selection, not automatic availability failover.
  • Local cost and energy values are estimates, not direct per-request measurements.
  • The cross-platform setup scripts automate detection and configuration, but their current migration command still names the removed fastapi Compose service after the backend/middleware split.
  • The repository currently has no first-party automated test suite, so the project should not be presented as production-hardened managed infrastructure.

The value of GPUShare is the system design: a usable interface and accounting layer around hardware that remains private, finite, and intermittently available. It is intentionally a trusted-group tool rather than a hyperscale cloud competitor.

The source is available at github.com/Slaymish/GPUShare.

Related Posts