There is evidence to suggest some LLM’s have the ability to produce original outputs, such as DeepMind’s solution to the cap set problem.
https://www.nature.com/articles/s41586-023-06924-6
On the other hand LLM’s have some incredible text compression abilities
https://arxiv.org/abs/2308.07633
I’m pretty sure there is copyright infringement going on by the letter of the law. But I also think the world would be better off if copyright laws were a bit more loose. Not wild-west anything-goes libertarianism, but more open than the current state.
I’d guess Stable Diffusion