If you count only the cost for you, maybe it doesn’t consume water, but your toy still guzzled lakes as it was training. Plus, the hardware to run a full sized LLM is expensive, so you bragging about how it costs nothing is like a millionaire preaching to gamblers that it’s better to just be rich than try to win at the slots
I specifically said “full sized”, a pc with modern gpu and more than 32gb of vram is not a regular computer that most gamers have access to. If you are running a 7B model on a gtx 1080 or even an rtx 3060, you are not running a full LLM like the ones you would get from a subscription service
It’s an analogy, it must be similar in principle, not in numbers. A subscription to chatgpt also costs less than what gamblers spend in slots. But whatever, I don’t care enough to argue much more
The very first comparison fails, though. I run LLMs locally on my own computer, tokens cost me nothing.
Every time I’m thirsty, I fill up my bathtub! Water costs me nothing!
I pay for my electricity. It uses roughly the same amount of power when I’m running an LLM as it would if I was playing a game. It’s negligible.
And contrary to all the breathless headlines about water-guzzling data centers, my computer doesn’t consume any water at all when I run an LLM.
If you count only the cost for you, maybe it doesn’t consume water, but your toy still guzzled lakes as it was training. Plus, the hardware to run a full sized LLM is expensive, so you bragging about how it costs nothing is like a millionaire preaching to gamblers that it’s better to just be rich than try to win at the slots
It’s a regular gaming PC. Are you going to dismiss all gamers as “millionaires”?
I specifically said “full sized”, a pc with modern gpu and more than 32gb of vram is not a regular computer that most gamers have access to. If you are running a 7B model on a gtx 1080 or even an rtx 3060, you are not running a full LLM like the ones you would get from a subscription service
Yes, I know. You’re saying that a 32GB graphics card is millionarie hardware? You’ve got a weird view of the cost of these things.
It’s an analogy, it must be similar in principle, not in numbers. A subscription to chatgpt also costs less than what gamblers spend in slots. But whatever, I don’t care enough to argue much more