06
Self-hosted GPU servers for AI models
Custom and open-source models served from GPU servers we run, in development and production, instead of paying a metered API per call.
- Role
- Full Stack Developer · Mavericks Tech Solutions
- When
- Sep 2025 – present
- Relevant for
- Cloud & InfrastructureAI & Agents
request path
- Your appsends a document
- Endpointprivate network · authenticated+9 ms
- GPU containercustom model · L4+380 ms
- ResponseJSON back to the app+12 ms
press send to trace one request
monthly cost · illustrative
Past the break-even point, every extra document on the private GPU is free.
My part
I provisioned the GPU instances, deployed the models in containers, set up SSH access and port tunnels, and fixed connectivity when an endpoint couldn't be reached.
Problem
Every extraction call was billed per request by a managed AI service, and documents left our network to be read. The cost grew with every document.
Approach
Provisioned GPU instances on Akamai Cloud for development and production, and on Google Cloud for evaluation. Packaged the models in containers, served them behind an authenticated private endpoint, and tunnelled ports so accuracy could be tested against the real serving path from a laptop.
Outcome
Models run on GPU servers the team controls, so cost no longer scales per call and documents stay in-house.
What I did
GPU instances
Provisioned GPU instances: Akamai Cloud for dev and production, Google Cloud for evaluation.
Akamai CloudGoogle Cloud
Two GPU types
NVIDIA L4 and RTX 6000 Pro run side by side.
Models in containers
Models packaged in Docker and served behind an authenticated private endpoint.
DockerLinux
SSH access and tunnels
SSH access and port tunnels so accuracy could be tested against the real serving path from a laptop.
SSH tunnels
Connectivity fixes
Fixed endpoints that couldn't be reached.
Firewall/ports
Stack
- Linux
- Docker
- SSH tunnels
- Firewall/ports
- NVIDIA L4
- RTX 6000 Pro
- Akamai Cloud
- Google Cloud