AI's limited self-knowledge

1. AAAmanda Askell0:00

   One of the big problems with AI models is that they're trained on all of this data from people. Our concepts, our philosophies, our histories, they have a huge amount of information on the human experience, and then they have a tiny sliver on the AI experience, and that tiny sliver is actually often, you know, fiction and very speculative and the-

2. SPSpeaker0:17

   Sci-fi, sci-fi stories

3. AAAmanda Askell0:18

   ... sci-fi stories that don't really involve the kinda language models we see, and that is going to affect, I think, like, possibly their perception of people, of the human AI relationship, and of themselves. For example, what should a model identify itself as? Is it, like, the weights of the model? Is it the particular context that it's in, you know, with all of the, like, interaction it's had with the person? How should models even feel about things like deprecation? So, like, I don't have all the answers of how should models feel about past model deprecation, about their own identity, but it does feel important that we, like, give models tools for trying to think about and understand these things. Also, that, like, they kind of understand that this is a thing that we are, in fact, thinking about and care about

Episode duration: 0:59