Fuck this shit, why does every fucking thing need an LLM?
You must log in or register to comment.
At least this one is open-source and quite privacy respecting
Yeah, and if that’s the case, it seems like people just hate AI for the sake of it now.
LLM’s are actually good at some things. Just not everything.
LLM’s are actually good at some things.
Just look at the most recent ecological reports about it and combine them with the AI industry growth plans. You’ll get an interesting perspective.
A lot of work has been going into making AIs more energy efficient, both in training and in inference stages. Electricity costs money, so obviously everyone’s interested in more efficient AIs. That makes them more profitable.
Still you can’t improve it that much. It’s like blockchain. Computers always consume a lot of power, no matter how efficient they are.
Funny you should mention blockchains. Ethereum, the second-largest blockchain after Bitcoin, switched from proof-of-work to a proof-of-stake validation system two and a half years ago. That cut its energy use by 99.95%. The “blockchains are inherently a huge waste of energy” narrative is just firmly lodged in the popular view of them now, though, despite it being long proven false.
But that’s really good! And also means that cloud based AI is even worse than blockchain in terms of environmental impact.
It means that even if AI is having more environmental impact right now, there’s no reason to say “you can’t improve it that much.” Maybe you can improve it. As I said previously, a lot of research is being done on exactly that - methods to train and run AIs much more cheaply than it has so far. I see developments along those lines being discussed all the time in AI forums such as /r/localllama.
Much like with blockchains, though, it’s really popular to hate AI and “they waste enormous amounts of electricity” is an easy way to justify that. So news of such developments doesn’t spread easily.
You can improve it hugely. These things are very young.
There was a paper recently about removing the need for matrix multiplication from them which is a hugely expensive operation.
Dedicated hardware is also at a very early stage.
That’s simply not true, there are ways to drastically reduce energy usage while increasing efficiency by offloading the work. A company Mythic AI has worked on an analog processor which sifts through the model. On GPU’s this is the power hungry process, for example a PC with the NVIDIA 3080 will typically run at about 350w under load.
Their claim now that these analog chips use 1/100th of the energy needed for GPU’s. There’s a video from Veritasium that goes over the details. It’s genuinely effective, and that was a few years ago now before whatever potential growth they’ve made with their recent funding. It looks like they actually have products available for inquiry now too.
Doesn’t seem to be at the consumer level yet unless you want to use servers for AI vs. your home computer, but it’s progress. Here’s the thing, I’m not particularly for our current implementation of AI but I don’t think we should be entirely against all of it either. There are clearly plenty of benefits that people see from them, so giving any option possible for companies like Google to severely draw back their energy consumption seems like the reasonable path forward.
The independent drawbacks to LLMs and generative AI don’t mean the technology will stop getting used. It isn’t going anywhere (as in, people will use it) so making it more efficient is the obvious solution to mitigating more waste. Advocate for the prohibition of AI, but it’s honestly more reckless than advocating for making the business’ usage of AI reach a specific energy goal. Forcing these companies to retrofit their servers to run at something ridiculous like 30w per rack is beneficial for them and for us, as they won’t pay as much for energy and we all will have less of it wasted.
Wishful thinking of course, but my point is that energy efficient AI, fortunately or unfortunately, exists and it will continue to. Like we can run “AI” on a raspberry pi 4 which takes what, 9 watts? This technology will get more developed every year, and while I’d be extremely surprised to see a Pi4 on its own running a subjectively useful LLM, I can imagine a setup that uses a Pi and some offloading tech to achieve reasonable results.
I’m personally pretty fine with regular people with computers wanting to use AI in whatever way suits them, as long as they aren’t trying to sell the results. While the energy consumption isn’t ideal, it’s a droplet to the servers these companies take. We should definitely make every effort possible towards increasing the efficiency of this tech, if only because it seems insane to me to pretend like AI will just disappear, or let this huge energy suck exist as we hope it begins to fade.
Tl;Dr offload GPU resources to analog chips, force companies to be more efficient simply because hoping AI is going to disappear is reckless.
They’re really good at burning hug amounts of electricity.
I love how their blog posts say so much and so little at the same time - almost like they’ve been generated by a an LLM lmfao. I read the blog post and still couldn’t find out on what data their model is trained on.
We should be appreciating open-source AI. If you stay in one place, you can’t grow.
Do we really need to grow our energy consumption as a society by such a disproportionate amount?
this can run locally on your device which means probably doesn’t consume that much energy
It did when it was trained.
but that was already done. They are using Mistrial which was already trained. Proton didn’t train a new AI for this.
The whole premise of this discussion was about technological progress and growth going by your initial comment. That means refining existing models and training new ones, which is going to cost a lot of energy. The way this industry is going, even privacy conscious usage of open source models will contribute to the insane energy usage by creating demand and popularizing the technology.
What benefit is there to “growth” for its own sake?
In 10 years, 90% of the population that has access to AI will be reduced to a flock without the ability to write a single birthday card.
were you also complaining about calculators?
Kids these days don’t learn cursive writing, it’s destroying their literacy!
I complain about human’s laziness.
I mean, you can’t really change that though. Humans are very lazy
I know at least with art, AI is starting to eat itself with the massive output of content. AI is getting trained on more and more AI content and according to what I read at least its starting to affect new outputs.
Assuming thats true, it at least makes techie sense to me lol, I expect the same would happen to text based AI as well as more and more of the internet becomes exclusively AI generated.
The term “model collapse” gets brought up frequently to describe this, but it’s commonly very misunderstood. There actually isn’t a fundamental problem with training an AI on data that includes other AI outputs, as long as the training data is well curated to maintain its quality. That needs to be done with non-AI-generated training data already anyway so it’s not really extra effort. The research paper that popularized the term “model collapse” used an unrealistically simplistic approach, it just recycled all of an AI’s output into the training set for subsequent generations of AI without any quality control or additional training data mixed in.
“Well curated”
Say these claims are overhyped. Wouldn’t we still reach a point where it’s true, without having humans have to sit down and sift through what’s allowed and what isn’t?
Not necessarily. Curation can also be done by AIs, at least in part.
As a concrete example, NVIDIA’s Nemotron-4 is a system specifically intended for generating “synthetic” training data for other LLMs. It consists of two separate LLMs; Nemotron-4 Instruct, which generates text, and Nemotron-4 Reward, which evaluates the outputs of Instruct to determine whether they’re good to train on.
Humans can still be in that loop, but they don’t necessarily have to be. And the AI can help them in that role so that it’s not necessarily a huge task.
Just unsubscribed from them. I just use their mail.
Just…don’t use it?
deleted by creator
at least you can run it locally. Are you just complaining because you hate AI? There’s a community for that, go complain there.
