I’m looking at starting a service that involves hosting a lot of LLM models, which are often going to be 16GB+ (compressed). I did a bit of searching for cloud storage providers with cheap egress, and the cheapest I could find is $0.01 per GB, which would still be $0.16+ per download.
How do sites like Huggingface or CivitAI do it? Lots of VC funding?
Lots of VC funding?
Probably. They might have gotten additional discounts off of the advertized price by talking with sales and committing to the service for a year or other ways.
R2 by cloudfare does not charge egress costs. It’s 0.015$/GB/month for storage. Read operations are 0.36$ per million.
I do have a hard time believing that they will remain this cheap though, but who knows.
Storj does it at 7 USD/TB. And there are providers that technically provide unlimited bandwidth, like Hetzner’s dedicated servers; they still have some abuse limits, but even working within the limits should make it much cheaper. This means custom engineering though.
