That’s not correct btw. AI is supposed to be creative and come up with new text/images/ideas. Even with perfect training data. That creativity means creativity. We want it to come up with new text out of thin air. And perfect training data is not going to change anything about it. We’d need to remove the ability to generate fictional stories and lots of other answers, too. Or come up with an entirely different approach.
I’ve heard people saying that before. But it’s not true. You can ask an AI to draw you an astronaut on a horse and it’ll do it despite never having seen such picture. (Now it has.) Same applies to LLMs. They come up with an answer to your exact question. Not a similar one it saw on Reddit before. That answer might be wrong (which is my point) but if you try it, you’ll regularly find it tries answering your questions and not different ones.
I’ve also tried some scifi storywriting with AI and there it becomes quite obvious that it’s able to apply things it knows from different contexts and apply that to my setting. Like ethics questions, basic physics and what character can and cannot do. Rough knowledge about how stories are written. You can tell it to do a plot twist an an arbitrary point and it’ll do. All of that is knowledge about (abstract) concepts and the ability to apply it to different contexts. Which is an important part of creativity.
And I’ve read papers where the scientists try to look inside of AI and they are able to spot abstract concepts like what a cat is in the weights. It’s fascinating how it works. And it turns out it’s not just regurgitating it’s training data. Which isn’t surprising because a lot of effort has been put into the computer science behind it to make AI more than that. And it’s also why they’re useful in the first place.
It’s able to apply those things because it’s read millions of sci-fi stories, and can make an educated guess. It’s also able to produce an image od an astronaut on a horse because it’s seen lots of images of astronauts and horses, and people sitting on horses, so it can once again make an educated guess. I don’t think it’s right to call that creativity.
Isn’t that like 60% of what creativity is? Art sometimes is about combining things in a new way. I mean it’s rare anyways that one genius comes up with an entirely new concept like scifi stories or pop art and invents that genre out of thin air. Most of the times also humans take something that already exists and build upon that. It’s not that far off here. And I doubt a human can draw a “rkbvrpoi” on a “wuqrkah” and not take inspiration from …anything.
I mean obviously there is something missing. Some human told it to draw that astronaut. So the whole artwork contains that original creativity that didn’t come from the AI. But I think it’s debatable wheter it could do it. This is only one specific example
The hallucinations will continue until the training data is absolutely perfect
That’s not correct btw. AI is supposed to be creative and come up with new text/images/ideas. Even with perfect training data. That creativity means creativity. We want it to come up with new text out of thin air. And perfect training data is not going to change anything about it. We’d need to remove the ability to generate fictional stories and lots of other answers, too. Or come up with an entirely different approach.
AI isn’t supposed to be creative, it’s isn’t even capable of that. It’s meant to min/max it’s evaluation criterion against a test dataset
It does this by regurgitating the training data associated with a given input as closely as possible
I’ve heard people saying that before. But it’s not true. You can ask an AI to draw you an astronaut on a horse and it’ll do it despite never having seen such picture. (Now it has.) Same applies to LLMs. They come up with an answer to your exact question. Not a similar one it saw on Reddit before. That answer might be wrong (which is my point) but if you try it, you’ll regularly find it tries answering your questions and not different ones.
I’ve also tried some scifi storywriting with AI and there it becomes quite obvious that it’s able to apply things it knows from different contexts and apply that to my setting. Like ethics questions, basic physics and what character can and cannot do. Rough knowledge about how stories are written. You can tell it to do a plot twist an an arbitrary point and it’ll do. All of that is knowledge about (abstract) concepts and the ability to apply it to different contexts. Which is an important part of creativity.
And I’ve read papers where the scientists try to look inside of AI and they are able to spot abstract concepts like what a cat is in the weights. It’s fascinating how it works. And it turns out it’s not just regurgitating it’s training data. Which isn’t surprising because a lot of effort has been put into the computer science behind it to make AI more than that. And it’s also why they’re useful in the first place.
It’s able to apply those things because it’s read millions of sci-fi stories, and can make an educated guess. It’s also able to produce an image od an astronaut on a horse because it’s seen lots of images of astronauts and horses, and people sitting on horses, so it can once again make an educated guess. I don’t think it’s right to call that creativity.
Isn’t that like 60% of what creativity is? Art sometimes is about combining things in a new way. I mean it’s rare anyways that one genius comes up with an entirely new concept like scifi stories or pop art and invents that genre out of thin air. Most of the times also humans take something that already exists and build upon that. It’s not that far off here. And I doubt a human can draw a “rkbvrpoi” on a “wuqrkah” and not take inspiration from …anything.
I mean obviously there is something missing. Some human told it to draw that astronaut. So the whole artwork contains that original creativity that didn’t come from the AI. But I think it’s debatable wheter it could do it. This is only one specific example