What could make AI conscious? with Wojciech Zaremba, co-founder of OpenAI

Wojciech joins us to talk the principles behind OpenAI, the Fermi paradox, and the future stages of developments in AGI. Made by Angelica Pan using W&B
Angelica Pan

Listen on these platforms

Apple Podcasts Spotify Google Podcasts YouTube Soundcloud

Guest Bio

Wojciech Zaremba is a co-founder of OpenAI, a research company dedicated to discovering and enacting the path to safe artificial general intelligence. He was also Head of Robotics, where his team developed general-purpose robots through new approaches to transfer learning, and taught robots complex behaviors.

Connect with Wojciech

Show Notes

Topics Covered

0:00 Sneak peek and intro
1:03 The people and principles behind OpenAI
6:31 The stages of future AI developments
13:42 The Fermi paradox
16:18 What drives Wojciech?
19:17 Thoughts on robotics
24:58 Dota and other projects at OpenAI
33:42 What would make an AI conscious?
41:31 How to be succeed in robotics

Links Discussed

Transcript

Note: Transcriptions are provided by a third-party service, and may contain some inaccuracies. Please submit any corrections to angelica@wandb.com. Thank you!
Wojciech:
We almost think about it as the safety of [an] airplane. There will be multiple layers into it. You could imagine that one layer might have to do with appropriate data filtering, or maybe then another layer has to do with injecting human feedback. And maybe some like a final, I'd say, destination by the model at the very end.
Lukas:
You're listening to Gradient Dissent, a show about machine learning in the real world and I'm your host Lukas Biewald. Today we're talking to Wojciech Zaremba who's one of the co-founders of OpenAI and he's worked on the robotics team through most of his time there, where he made the hand that manipulated and solved a Rubik's cube. And at Weights & Biases we have been working with him for quite a long time and rooting for his team. So I'm super excited to get tactical on robotics, but also Wojciech loves to think deeply about the bigger picture in AI and so we'll get into that too. The first question I would ask you about was what it was like starting OpenAI.
Wojciech:
The first time I heard the idea when I met with Greg¹ in New York, and actually even when I was about to meet the first time I overslept and we were about to meet at 5:00 PM. And then I have this like a weird working schedule that I used to do research over the night. And I was going to sleep like at 6:00 AM, 7:00 AM. So I overslept for our meeting at 5:00 PM, but eventually we met. I would say early on, there was some discussion about the mission of the company. It's also interesting that back then in the community whenever someone spoke about safety, they were considered pretty much crazy. I mean, people are saying, "Oh, AI is so far away that it actually makes no sense to speak about it." There were even these quotes saying that it's like thinking about overpopulation on Mars, and at some point that might be a problem, but we shouldn't be concerned about it today. Yeah. So I was also very excited like I told Greg that one of the most important people that we have to have is Ilya². And we've got Ilya. Then there was a meeting around November 2015 in Napa. I met there with Sam³, I met there with Greg, Ilya, John Schulman⁴. There was also Andrej⁵ who is now at Tesla. And of course we discussed AGI. What are the steps? What do we think is missing? It was also, I could see that these folks are thinking about, like even during spare time, about big fundamental questions. So there was this time that we're sitting at the table and Sam Altman asked everyone what they think what's the solution to Fermi Paradox, why we don't observe aliens? And the people had very sophisticated opinion about this topic. And I was thinking, "Oh, that's the group of people with whom I would like to work. They consider this even like a metaphysical questions." Because it's almost like the questions about AGI are almost like a metaphysical.
1: Greg Brockman, co-founder and CTO at OpenAI
2: Ilya Sutskever, co-founder and Chief Scientist at OpenAI
3: Sam Altman, CEO of OpenAI
4: John Schulma, co-founder and research scientist at OpenAI
5: Andrej Karpathy, Senior Director of AI at Tesla
Lukas:
That makes sense. But I guess what's intriguing about the way OpenAI operates is at its core, its mission is focused on AI safety I believe. But it seems like the remarkable results coming out of OpenAI, including the stuff that you work on is it seems less about safety and more about showing the power of AI or moving the field forward. Is that right or what's the thinking there?
Wojciech:
So we have plenty of teams working on safety and one of the efforts working on safety was "Let's try to foresight what it takes to build AGI." So people started to look from perspective of resources, how the things actually scale and connect. And then they realized that we are able to make our models significantly better with appropriate scale-up. And in some sense, that was actually a result of safety work. Another example maybe like that is there has been work on human feedback. So the idea is "How could we inject human values into the model? How could we tell model what is good versus what is bad?" Then of course at first people started to work in the academic, most simplified domain. So the question is "How can we tell model that the given summary of text is good versus bad?" It turns out that actually these developments led to capabilities, but motivation was from safety perspective. And our stance regarding safety is we would wish to maximally release, let's say, capabilities and description, how to build safe systems. But it's also very likely that it might be that safety and capabilities may be the same thing. In some sense, safety means "What to do to make sure that we can control the model". So there's various levels of safety. One level of safety is "What to do to make sure that we can control the model". And this is also very similar like from perspective of commercialization or capabilities, that's also what you want to happen. You don't want the models that go nuts when you are asking some slightly out of distribution question. So when you think from perspective of our mission, the mission is to serve humanity and there are actually three different axes how you can distribute what we have developed. One axis is you can literally just give people money, something like universal basic income. I mean, that still requires actually making a lot of money to make any difference to people. Second one is you can give away technology. So we are, let's say, building technology and we are actually sharing it maximally. And third one is governance. So the question is "How to make sure that the humanity as a whole can decide on what to do with this technology?" And OpenAI actually is interested in each of these axes and they're like at various stages.
Lukas:
I'm kind of curious, do you fear AGI? You talked about Fermi's Paradox and it seems like one reason that we don't see aliens might be that they developed AGI or some technology and killed themselves inevitably, right? That could be one reason that we don't see them. Do you worry about that? Do you put a percent probability on that? Is that something you imagine might happen in your lifetime?
Wojciech:
Yes, I think it's possible, but I think that there will be actually various stages of AI developments. The first stage is when AI will become very valuable commercially, and I believe that might be multi-trillion industry. Then second stage is actually AI might become a national security threat. So you could imagine that AI could be used to control a firm of bots or manipulate elderly or sway public opinion for some election or so. In some sense, you can say that it's already happening in some format, that there is selectively displayed content online that actually biases people in various ways.
Lukas:
Yeah.
Wojciech:
The first stage is essentially that the value of technology is just keep on increasing, second stage it's national security and then the third stage is existential risk to humanity. It's almost the question [of] how they are spreading in time and so on. And usually we should just be worried [about] the initial parts of the sequence, and we should bear in mind all the pieces. We shouldn't just focus on the last one.
Lukas:
I see. So we should focus on all three of those risks then?
Wojciech:
Correct. Or like, the first one, the risk is not an increase of commercial value, I guess maybe the risk might be job misplacement.
Lukas:
Right. I mean, do you have a sense for yourself a probability that you put on existential risk?
Wojciech:
It's actually hard for me to think here in terms of probabilities. I could tell you some convincing stories and it's also, I noticed that these probabilities they really change over the time depending on some external factors and so on.
Lukas:
What external factors change your probabilities? Because we're not really getting new information, right?
Wojciech:
Yeah. So I'm saying external factors like political climate or so.
Lukas:
Ah, I see.
Wojciech:
Let's see. Let me tell you the gloom story and I can tell you, let's say, positive story.
Lukas:
Okay, great. Let's start with the gloom and then do the positive one.
Wojciech:
In principle you can say that it's almost an inevitable that we'll build superhuman AI. It's just a matter of time. Then it's also very likely that we'll end up actually with multiple organizations building it. Because it's so valuable and there will be a competition. There might be some organization ahead, but it's very likely that we'll end up with multiple organizations. Then there will be, various people will be tinkering with the code of AI and AI will be tinkering with its own code. And it will have a powerful capabilities to achieve various goals. Initially, this would be goals given by a human. But then [we] can notice that at least in case of natural organisms that also are derived from code, that's DNA code, there is this property that if you slightly mess up the code, it actually, the organism might misbehave. It actually might work against the host. So in case of cells, it's actually possible to get a cancer. And cancer is a prevalent phenomenon in the nature. So then you could imagine now in case of AIs, maybe if you have a couple of AIs, then we actually know what they are optimizing for and who they serve. But once then there is a increased number of AIs and in some sense there's a process of mutation, which is AIs are modifying its own code, humans are modifying their own code. Then there is a process of natural selection. And you can say that the AI that literally wants to maximally spread will be the one that will exist. The things in the universe that want to replicate are the things that exist. Here the main difference is that AI will have just huge power, therefore it's kind of risky. What are the consequences of AI wanting really just to optimize for application? So I guess that's maybe a gloom scenario.
Lukas:
One question I always have about the gloom scenario, I mean it makes sense to me but I feel like the metaphor of natural selection...well, at least with plants and animals we reproduce, right? So like you can't change the whole system at once, but it seems like AI might have a more complicated system of changing and reproduction. Like you could imagine all the AIs changing at once or communicating. It seems like you might not necessarily...you could imagine a stable equilibrium, right? Where things aren't allowed to consume other resources for example, right? Or is there, am I missing something?
Wojciech:
It's possible that we'll have thousands of benign AIs and it might be not that simple even, to get all the resources, but you could imagine that randomly happens. So that one of the AIs won't be that benign. And it happened because people are modifying code, plus it started optimizing different reward function, and it still has immense skills and then it can pursue its goal. Then it might be like I said, other AIs are defending the system or maybe they were never trained for defending. It's very hard to predict the dynamics in "remove the agent" setup. With one AI you can maybe predict what are the possibilities. It would be still extremely hard, but once you have many of them competing in some sense for resources, very hard to say actually what might be the consequences.
Lukas:
Okay. So tell me the positive story.
Wojciech:
Can say that even if AI would become so powerful, it wouldn't even care that much to be here. It'll just go to the stars. It would build all sorts of technology for us. It's like the same way as we are not competing with crystals. Crystals are also replicating. It's like a self-replicating machinery. It's kind of in different level of abstraction, and it doesn't bother us that they are replicating. Of course, there's all the advancements that could happen. Could imagine that AI would cure all the diseases, remove suffering, allow us to go to the stars, so on.
Lukas:
It's interesting though in both those scenarios, it involves steadily consuming resources and expanding. It's just in one, the AI leaves us alone and the other it doesn't care or maybe it consumes our planet. But in both cases, wouldn't you think that we would see evidence of this in some other alien life that created an AI and came to us in some self-replicating way? What do you think about that?
Wojciech:
You're asking the question about Fermi Paradox.
Lukas:
Yes. Sorry. You brought it up, so it's top of mind. Isn't there a collapsed scenario, I guess.
Wojciech:
Let's say you said, "Oh, if aliens would build AGI and then AGI destroyed them, but then we would see some traces of AGI in the universe. Like the AGI would consume a lot of resources, assuming that actually...." So there's a few assumptions. There's assumption that once you are sufficiently technologically advanced, then you're spreading in every direction in the universe with the speed of light. And we haven't observed in any parts of universe, anything like that. We haven't seen any Dyson spheres or so. One simple explanation might be that actually we are alone in the universe. Maybe it's so unlikely for life to flourish that we are alone. So that almost puts maybe more responsibility, but who knows.
Lukas:
Is that what you believe?
Wojciech:
I have a probability distribution over beliefs, what might be the case.
Lukas:
Tell me...you can't reveal your distribution?
Wojciech:
So let's see. I can tell you a fun one that I heard recently. Let's say you are having super advanced civilization then of course, it makes sense to turn the entire planet into computer and then to maximally use matter for the purpose of computation. One thing that is actually interesting is apparently once the universe would be cooler, then it is possible to do more efficient computation. So one statement is that maybe aliens are just waiting for universe be cooler. But I'm not sure if I believe in this, like it might be cool description.
Lukas:
So I guess, how do these beliefs inform the work that you do? Like you talked about two kind of bad AI scenarios that both actually seem very relevant to me. I feel like the inequality feels real to me right now at this moment, and the political stuff also feels like it's starting to become real. And then the existential threat feels like you're telling me a very compelling story, but somehow it doesn't feel the same visceral fear for me and my child. But maybe that's irrational. How do you think about...are those three worries what really drives you to do your work? Or are they more theoretical for you and how do you weight the different AI safety issues?
Wojciech:
Actually, let me at first try to even describe where usually the drive comes from. As a kid, I did quite a lot of mathematics. And you realize that in mathematics, I've got a lot of pleasure by solving difficult problems. That all of a sudden, like this amazing moment of excitement once I was able to figure out a solution to some mathematical problem. I actually realized that that's the main drive for majority of scientists. That there is a just very complicated puzzle involving mathematics and computers, and somehow they can put all the pieces together, and that actually gives them amazing excitement. So that's cool. But simultaneously, it would be very sad if due to these excitement, we would actually destroy a lot of value or destroy how the humans operate and so on. So there is a piece of me that is excited about the technology, about solving mathematical and computer science problems. And there is also a part of me, like I'm thinking maybe from perspective of altruism and responsibility. It's like at some point of my life I realized that ultimately the happiness comes from within and I actually have already everything that I need. Then it's almost like my cup is full, just want to make sure that there is enough for others. So then it becomes quite natural to think, "How can I actually make sure that my work has the maximally positive impact?" And in case of AI, it is actually quite complicated.
Lukas:
Why did you choose to work on robotics?
Wojciech:
Actually, here is a reveal. I was actually working for several years on robotics and as of recently we changed the focus at OpenAI, and I'm actually, I disbanded the robotics team.
Lukas:
Oh, wow.
Wojciech:
Yeah.
Lukas:
Why did you do that?
Wojciech:
Okay. So, the reasoning is...there is a few pieces. It turns out that we can make or check on the progress whenever we have access to data. And I kept all our machinery unsupervised, reinforcement learning, they work extremely well. There is actually plenty of domains that are very, very rich with data. And ultimately that was holding us back in case of robotics. This decision was quite hard for me. I got the realization sometime ago that actually that's the best from perspective of the company. The sad thing is, I think if we would be a robotics company, or if the mission of the company would be different then I think we would just continue. I actually quite strongly believe in the approach that robotics took and the direction. But from perspective of what we want to achieve, which is to build AGI, I think there was actually some components missing. So when we created robotics, we thought that we can go very far with self-generated data and reinforcement learning. At the moment, I believe that actually pre-training allows to give model 100X cheaper IQ points, and then that might be followed with other techniques.
Lukas:
And what is pre-training?
Wojciech:
Pre-training, that's like, I can explain it in case of GPT-3. So pre-training in case of GPT-3, or in case of language models, means training them on some unsupervised task, such as next word prediction. And that builds in all the internal representation that allows the model to off the bat solve many tasks. And in case of robotics we haven't had such a data.
Lukas:
I see. So do you regret working on robotics?
Wojciech:
No. I think that actually we've got plenty of insights for other projects. I think that also we built a really amazing technology. I would say I'm actually very proud. There was like of course moments of sadness when I was making this decision, but I'm quite happy where we've got. Also even from my own perspective, in the meanwhile I manage also other teams. That made some significant progress in the new world and more information, there will be more information about it sometime.
Lukas:
Cool. I guess one thing that I always observe is when you look at what computers do versus what seems easy, robotics seems the most striking. I feel like the simplest things of picking up an arbitrary object, it seems like the most natural thing for my brain. It seems so hard, maybe harder than anything else that feels natural, to make a robot do it. What do you think about that? Do you think that there's more progress in the short term or will it be the last thing that we solve on the path to AGI?
Wojciech:
So there are two possibilities for me, like a few possibilities. So one is if someone would be able to actually in a natural way to collect a lot of data, I think that might be the capabilities. Another possibility is that we just need very powerful video models, the same way as at the moment we have very powerful text models. We need very powerful video models to take it off the ground. The trickiness at the moment with video models is that they just require way more compute than text models. So in case of text, already individual word conveys a lot, a lot of information and it just takes few bits to represent it. In case of video, if we would like to process images of a size few hundred by few hundred several frames at a time, that requires orders of magnitude more compute. I believe that if we would have models that have a really powerful understanding of video, it would be way easier to train them toward manipulation. There is also one more technical issue here. It's like, these small models most likely, they would have to be very huge and then the difficulties in running them real time. So at the moment I see a few issues with robotics simultaneously, and this idea to be able to go after domains when the number of issues is like, let's say one or two is very favorable. It's also when we started...okay, in some sense, we started all sorts of projects at the beginning of OpenAI and we haven't had the clarity how and exactly what we want to build. And over the time, we got way more clarity and the amount we can increase the focus in different directions.
Lukas:
So that's the other question that I've always had, how does OpenAI think about the projects you pick? I feel like, maybe critics would say that OpenAI has sort of been too good at picking projects that are very evocative. Like you guys put out these GPT-3 and the music stuff that you did, like at least to me it just seems so cool. But I think maybe some people feel frustrated that it's like, it feels almost targeted towards like a media event or something. Is that something that you think about at OpenAI or I guess, how does OpenAI pick what to work on next?
Wojciech:
We have some internal beliefs, what has to be built for general purpose intelligence. And people mostly choose projects on their own. There is also, let's say, there is some level of freedom to go after crazy high-payoff ideas. I don't think ever that people are like saying, "Let's go after this one because it's high PR payoff." It's more that we have amazing people in conveying our work to public. And maybe if we would release a GPT-3 or Jukebox as TXT file, then people wouldn't say that it was for, that they wouldn't say such things.
Lukas:
If you just did a bad job with the PR, the people would give you more benefit of the doubt. But I don't know, I feel like you chose to win Dota which...weren't other people thinking about this and it seemed like it was a very clear milestone I guess, as opposed to putting out a paper on reinforcement learning at massive scale or something like that.
Wojciech:
Yeah. So there's also actually element of internal motivation with these significant goals. I actually, I think that Elon suggested us to go after Dota. Motivation was, "Let's pick very complicated game," such that if we would make a progress, it would be undeniable. So there is a lot of toy tasks out there. Like for instance, people work on a humanoid walking in MuJoCo and this one is clearly I'd say disconnected from reality. Because people can make it walk in a simulation for multiple years already, but none of it works in reality. And then here in case of Dota, we wanted to ensure that actually what we are after, it's meaningful. So, how to ensure that it's meaningful? Some people are really devoting their life to actually play Dota, who are strategizing about it to play against us.
Lukas:
How much of the work then on Dota was, you felt, like fundamentally moving ML forward and how much of it was Dota-specific or can you even pull those apart?
Wojciech:
I think there was a decent amount of Dota-specific work. And then I think it was more than optimal, but also simultaneously hard. So I remember at the beginning of Dota project, it was actually unclear how to approach it. People are saying that contemporary reinforcement learning will have no chance in solving this problem. And people looked into off policy matters, on policy matters, evolutionary strategies. The thing that became quite surprising is that methods that already exist, with appropriate scale work extremely well. So that was a big surprise. And I remember some people even before Dota time at OpenAI, saying that maybe reinforcement learning is a dead end. And all of a sudden it's a very different story now.
Lukas:
For sure. At OpenAI, do you feel like you're competing with someone?
Wojciech:
The way how I would like the competition to be fully perceived is actually a competition with bad outcome.
Lukas:
With what?
Wojciech:
Bad outcome.
Lukas:
Bad outcome?
Wojciech:
Mm-hmm (affirmative).
Lukas:
Oh, I see. Competing with a bad outcome.
Wojciech:
I wouldn't like us to necessarily compete against let's say other technical labs and so on, but obviously there is some fear of being scooped or so. It's interesting that in case of large projects, I have seen it the way less than in case of work of individuals on a paper. So my understanding is that it's very easy to be scooped when you're working alone. And it's almost like impossible to get scooped if you work with, let's say, seven people.
Lukas:
Why is that?
Wojciech:
So I think it might have to do with that there is many people working individually, but very few working as a group.
Lukas:
It does seem like OpenAI is maybe uniquely good at that. It seems like compared to academia, you have much more authors on your...or compared to ML research typically you seem to do bigger projects and have more authors on your papers.
Wojciech:
I think that in reality we need both. Sometimes we need these insights from secluded individuals who are working [from] their hermit house for several months to figure out that there is actually a different way to build a Transformer or to train models or so. And it's almost impossible to work on such stuff as a larger group. But then eventually we want to build systems. The systems allow us to all [at] the same [time], to take our work to next level, next level, next level.
Lukas:
I guess. What role do you feel like OpenAI plays that maybe the corporate, like DeepMind isn't doing or Berkeley isn't doing?
Wojciech:
I actually think that OpenAI has fair amount of push on safety, that it became a mainstream topic. It wasn't a mainstream topic. So I think that's extremely important. Yeah. I actually think that's one of the most important things.
Lukas:
Do you feel like it's sufficiently a mainstream topic now? I mean, it's certainly much more mainstream than 2015.
Wojciech:
In some sense, I would like it to be sufficiently mainstream such that we would avoid bad outcomes. But I also almost think that the small bad outcomes might be a good thing. Because then they will inform the public that actually, these problems are real rather than imaginary. At the moment in case of GPT-3, we see some rudimentary aspects of safety. It's more like on the side of controllability. We have a model that can have a conversation with you, but it's unclear how to make sure that the model won't be offending you or won't go off the track or won't leak some secret information. And we almost think about it as the safety of airplane. There will be multiple layers into it. Like I could imagine that one layer I'd have to do with appropriate data filtering, or maybe then another layer has to do with injecting human feedback and maybe some final, let's say, discrimination item at the very end. So I would say, I think that at OpenAI there is a lot of discussion about this topic. And at the moment, some aspects of safety, they became even important from commercial perspective.
Lukas:
Mm-hmm (affirmative). And so it seems like you've made GPT-3 something of a commercial product. Is that right? Is that how you think about it?
Wojciech:
Yes. I mean, our thinking is that if we want actually to deploy AGI one day, then it actually might be very important to have a lower stake around before. And GPT, it's definitely lower stake. We can see what are the ways how the systems might be failing?
Lukas:
Do you think there's any intuitions from neuroscience in general that can guide the development of machine learning models?
Wojciech:
There is obviously a question, is consciousness independent of intelligence? Or how they are related and what would it make AI conscious? I guess they're like a few proposals now. It might be the case that all what is needed to be conscious, is to build a model of reality around. And at the moment, our models, they implicitly build such a model. That would be a claim in direction that actually our models are conscious. There is maybe, that's maybe one axis. Other axis is another idea behind what consciousness could be. It's like, you can look in mathematics and computer science for some very special mathematical objects. You can notice that in mathematics, there is a lot of weird things pop up once you allow mathematical system to be powerful enough to point on itself. In computer science, there is similar phenomenon with halting problem. Once the system points on itself, there is undecisiveness. I can say that maybe intelligence fundamentally has to do with compression, and compression and prediction are the same thing, so for instance, next frame prediction is actually compression. And once the system would become powerful enough that it tries to compress itself, that might be some in some way analogous to halting problem or to get the L Theorem (?) in mathematics. Also some people claim that consciousness is not a property of information, but rather it's a physical property, most likely of electromagnetic field. Then that would actually mean that our AI wouldn't be conscious. It could have the same behavior as we do, but it wouldn't be conscious. So I frankly don't know which of this is true and that's something that I actually keep on thinking about, I'll say a fair amount, because in some sense, consciousness is almost our subjective experience. It's almost the only thing that I can be certain about. When I wake up, that's something that I experienced. I cannot be that certain about mathematical equation or that tomorrow there will be a new day, but I'm certain that I'm having conscious experience at the moment. So it is an incredible mystery and I think it should be solvable by science. And AI allows to...or in case of artificial intelligence systems, we can control every aspect of the computation.
Lukas:
I guess, one difference with consciousness and the halting problem maybe, there's not a binary consciousness on versus off, but it seems to me like there's different levels of this. I think we sort of intuit that in the sense that we want to be kind towards other humans and we want to be somewhat kind to a cat, but we don't put it on the same level. Do you feel like the models you're building might be sort of approaching the consciousness of a worm or something? I mean, certainly they can do things that animals can't do.
Wojciech:
So yeah, I frankly I don't know. There is a Slack channel at OpenAI about welfare for artificial intelligence. Because it is conceivable that through some kinds of trainings, we could generate immense amount of suffering like massive genocides, but frankly, we don't understand it. We don't know if let's say giving negative reward to model is the same as stabbing someone.
Lukas:
Right. It seems at first glance it seems maybe ridiculous, but then it's hard to pull it apart. It's hard to really articulate what the difference is.
Wojciech:
Yeah. I mean, the interesting thing is... So I can see now path from here to AGI. Of course it might take a really long time and people are like, I think that there is a belief maybe that if model would be having human intelligence, then most likely it would be as conscious as a human. At the same time, at the moment, I can speak with GPT. I can ask GPT about consciousness, and it would tell me, "Yeah, of course." It would explain its conscious state and so on. Of course it has to do with GPT being trained with data speaking about the consciousness. But the weird thing is, how would I be able to distinguish if indeed GPT would become conscious, versus just knowing about it? I think there's a few funny answers that come to my mind. So one is we could try to remove all the data that matches consciousness, train model on it, and then have a conversation about consciousness and the model will say, "Oh, that's something I was thinking about. And I noticed this thing and that's so surprising that it's there." That would be maybe one way. Another way that comes to my mind has to do with even how to check that some other human is conscious. So one idea of verifying that some other human is conscious, is literally by connecting brains. If you can connect brains and feel that their consciousness expanded, then that might be an indication that someone else is conscious. There are of course various counterexamples, but you could imagine that similarity, if you would connect your brain to AI, and if you would experience that your consciousness expanded, that might be an evidence.
Lukas:
Well, that might be a nice note to end on, but I do want to pull this back into a little bit more practical realm for two final questions that we always ask people. The second-to-last question we always ask is, what's a topic in machine learning right now that you think is underrated or doesn't have enough people paying attention to it? Maybe something that you would study if you were totally free to start anew on some other topic.
Wojciech:
I actually think that I think that the models that can decide on its own compute budget, that they can keep on spinning inside like a Turing-complete model, like a universal Turing machine or a universal Transformer. Or you can think about something like having inner monologue as a means of just increasing amount of compute, that it's the model somehow, while solving problem it speaks inside of its head. I think that's what I would work on.
Lukas:
Cool. All right. The last question that we always ask, and this is for our audience which I think is a little more practically minded day-to-day than the conversation we got into, but what's the thing that you think is the hardest part today of going from a conceived model to a deployed model? And maybe specifically for you, I'm curious in robotics. If you were building a robotics company or OpenAI was like geared towards just making a successful robotics application, which would be amazing, what do you think are the challenges that you need to solve today to make that work?
Wojciech:
So I think that there are actually two stages. So first stage is creating a model that is good enough for any deployment. And then second one is literally building meaningful, viable products such that there is feedback and actually resources can be focused in that appropriate place.
Lukas:
And what might that look like? So you need something useful enough that you could make a lot of it and deploy it so it's collecting data that...am I understanding you right?
Wojciech:
Yeah. So, I mean, you could imagine for instance for a robotics company, seems to me that the problem of beacon place is actually completely tractable. I would also say that I wouldn't shy away from collecting data. So I think that the path that I would take now, if I would be focused on solving the problem, I would at first try to find some viable domain where there's a big enough market, the movement doesn't look complicated enough. And then I would ask team, hire plenty of people to parallel operation. And I would collect million trajectories and then train a model on it. And I would say people are very excited about reinforcement learning. And I think reinforcement learning is very, very powerful. And while the same time, I'll say they shy away almost from supervised learning. In my belief, if I would have a company I would double down on supervised learning and it's just keep on surprising me how far it takes.
Lukas:
All right. Well, thank you so much that was a lot of fun, I really appreciate you getting up so early.
Wojciech:
Thank you Lukas. Well, have a great day.
Lukas:
Thanks for listening to another episode of Gradient Dissent. Doing these interviews are a lot of fun and it's especially fun for me when I can actually hear from the people that are listening to these episodes. So if you wouldn't mind leaving a comment and telling me what you think, or starting a conversation that would make me inspired to do more of these episodes. And also if you wouldn't mind liking and subscribing, I'd appreciate that a lot.