Hey. Okay. All right, so thanks for joining us for our fifth session of The Anthem Poll, the future of human security. We'll start with Nick. So, Nick, you there? Okay. Yeah, so we're off? Oh, yeah. Yes, we're off. Okay, so as agreed, I hope, last week, when we're sort of hit peak insanity with Roko's Basilisk, and are now going to systematically work our way back down the slope, revisiting
the topics in reverse sequence, and ultimately getting back to the question of intelligence explosion right at the end. So I've got really three little introductory remarks to make this time that I hope will sort of help to get things rolling. First, moving towards the AI box experiment from Roko's Basilisk down through Newcombe's Paradox to the AI box experiment.
And then talking a little bit about the time structures that are coming into this material. And thirdly, which might be the best thing actually to start with, is a little bit of a digressive element, but I think it's suggestive in a way that might be picked up on and connects with other stars. So one of the things that Yudkowsky takes from Newcombe's paradox is what leads him to his maxim that rationality is systematized winning. I see that in the same way as no
one foreign to geometry enters here above Plato's academy. It's a kind of an anthropole slogan. I'm assuming were anthropole to be set up, its brief would be definitely to win. any definition of rationality that it came up with would be within that context. So I thought there was an extremely interesting thing came up which is to do with the chess match. I'm sorry, I'm just going to post a link for you guys. in case you haven't seen this it's about
um Kasparov's famous test match with Deep Blue the first time that a computer or algorithms could be said to be the world chess champion was by defeating Kasparov in this series of chess matches it was very tightly fought and Kasparov famously said that at a certain point in his matches against the blue he caught an intimation of an alien intelligence in this chess program. Now what is interesting coming out of this new material, there's a little video attached to the
link that I've put, is that it looks now as if I'm deep blue actually glitched out at this crucial point in the match is described as being caught in a loop that the program actually couldn't make a a move that it was kinda I'm content about and ended up just but basically making time-wasting move just in order to break its way out of this this loop it was caught in and and cat the cacus most my man away or he was seeing this alien intelligence corresponded to a moment where this chest program is actually
fritzing out its doing something that doesn't make sense to him and didn't make sense actually to the program and itself and was a move that was kind of random and and so Kasper projects onto this this this intimation of an alien intelligence now there's something I think quite deep about this it's tied up with a lot of other parallel phenomena and but I think just to take away the media take away is just this
a element that can purely chance mistake a glitch actually plays this role but if we go back to the Kowski comment context you can start up is what wins you know deeply wins this chess match if rationality is systematized winning this is what actually counts as the most exalted moment of rationality in the in the deep blue program and it was just a mistake it was it was you know by as far as the programmers were concerned as far as all the kind of rational context and background to this concern this is an error
it's a software failure and it actually produces this winning opportunity uh... so castro is defeated by mistake by software uh... say that's i'm just gonna leave that and little box on its own but it seems to me extremely suggestive in in various ways uh... soon now uh... let me i've just gone to time this site at go on to out this uh... maybe i'll just quickly run through the time stuff because i think that that could become huge and and and we don't want it to necessarily at this
point obviously people can't about it uh... but i think that the point to start with that is in the whole popular call culture surround this question of AI x-risk, there's a huge role played by the Terminator franchise. And I think people almost now have stopped seeing it as a question. Why should it be that within that particular myth, the great popular myth of this menace, that there there is simultaneously the question of the arrival of artificial intelligence and time disturbance. Why is it a franchise that is simultaneously about time travel and about
artificial intelligence? Now one could obviously take that just as a complete random coincidence or two things have just been kind of glued together without any deep motivation there. But I think that we can already see on the path we're taking sort of down the slope from Roko's Basilisk that there are reasons to think that there are connections between these two topics, if they are indeed two topics. And I've sort of subdivided them into three pointers that are all, I think, independent to a certain extent.
And going from, you know, following our downslope path, the most crazy of these connections is obviously what you see with Rocco's Basilisk itself. Because Rocco's Basilisk notion of a causal trade is the possibility of a negotiation with an entity in the future, as we were saying last week. You can engage in a causal trade with something that is infinitely distant from you in space or time. And obviously, in the context that works with the particular the Rokos Basilisk model, which is that this is a future artificial intelligence with which one is being involved, then that reduces, that collapses this kind of infinite possibility
of distance to something more specific, something more like the Terminator model, that there is a communication taking place from its point of view backwards across time. But Roko's Basilisk is a mode of retro-chronic incursion from the position of a future artificial intelligent entity. So that obviously is one way in which this gets, I think, quite explicitly set up. The second, a little bit more subtle, is tied up with Newcombe's paradox. And we go, I think, uncontroversially we're going down the slope to the Newcombe's paradox.
Newcombe's paradox is taken seriously much more widely than Rocco's Basilisk. I think that's totally, one can say that with great confidence. And that it's taken seriously, I think, comes from the fact it was publicized by Robert Nozick. That's where it really receives sort of general attention. And he himself comments that everyone thinks the basic problem posed by Newcomb's paradox has a simple solution, but people divide into two roughly equal camps about what that sensible, obvious solution is. it's you know there's a 50-50 split between using
language we we introduced last time between whether you're gonna accept a causal decision theory or an evidential decision whether in the specific terms of Newcomb's Paradox you're going to be a one boxer or two boxer as Nosek says you know people fall into these two different camps and each of them think that the other side is being clearly irrational and so the reason the fact that there's that division fact that there's just total lack of consensus about what a rational strategy is in this situation is I think the reason it gets taken seriously people are forced to take it seriously because there's such a disagreement
how to solve it it can't be neatly shelved by people just settling on one or the other side this time and and I think Newcomer's paradox as a whole can be constructed quite straightforwardly as a time travel simulation it explicitly but the basic problem as you remember I hope is that and Omega as the less wrong people call it or the predictor as it's originally called has either put a million dollars in this closed box or not and at the time that you are actually making the decision
do you open both boxes or one box its closed and it's going to be no other activity taking place in terms of that box at all it's over that's why the causal decision people say look Omega has either put a million bucks in there or not so why so it call take it away and if we decide to open the other box as well take the thousand box not of course the evidential decision say I'm but the record shows that every time people do that to boxing almost every time a day they lose say by can you think asking action rationality system I swimming you you go for one box it just cuz that works it if you don't understand why. So this prediction or prophecy that Omegari is supposed to take
place, I think is perfectly isomorphic with a retro chronic action. There's no difference ultimately in the structure of something about perfect prediction and ability to influence the past an action of the future upon the past can be for later as prophetic or reliably predictive act because if something from the future about which you are making prediction or prophecy uh... is confidently held to be going to occur then that thing is operating through you through the prediction now
and that's the they can be just converted into each other perfect perfect we got perfect confidence those two things so and new comes parks isn't set up as a time travel story but if you were to set up as one you would not have to trip at all and I'll look at it could you could just set the whole thing up saying Omega is able retro chronically to insert or remove a million bucks from the closed box. The only reason that you wouldn't put it that way is because that would make the causal decision theory approach, the two box approach, completely and obviously irrational.
It wouldn't be a serious question at that point. but if you're a hardcore one boxer then there's no difference between your models in those two cases if you say the reason I'm taking only one box is because I trust omegas predictive capability and if I take two box I know it will have predicted that it will not be a million dollars in the closed box or if you say a up Omega has variability and it can from the future change whether or not there's a million dollars in the box. Those two things are exactly the same once you have taken the one-boxer interpretation of the situation. So that's the second way that time disturbance gets moved into this.
And the third most sensible and I think simply undeniable, crucial, we'll come back to it a lot in what we're going to be dealing with and I think in future weeks, is just preemption. As we'll see, I won't go into this in detail now because we'll be dealing with it in the AI box experiment, but it's absolutely fundamental to Yudkowsky's position that you cannot, his whole system of argument is set up to prove that you cannot deal with this threat in, so to speak, real time. At the point that it's there, at the point where you have created
a superintelligence, it is too late to deal with it. There is no possible strategy that will work or see how he tries to argue that. And that means that you have to deal with it in advance. So it's an absolute fundamental strategic tenet of the anthropological orientation insofar as Yudkowsky has anything to do with setting that up, that it has to be preemptive. it has to deal with this problem it up of it becoming actualized and obviously that tends towards a cascade once you once you are in a sort of
a competition for preemptive action there's no natural equilibrium point you always want to get back earlier and earlier if starting from now you want to be faster and faster and faster addressing the problem but once it slips by another stage you start looking into history and start looking into the past and start actually analyzing things in terms of what has already it perhaps in some deeply suck tissues a camouflage way already taken place you know the pressure is always to have started at the earliest possible opportunity and so something that happened already is more dangerous and something but it's gonna happen
in the future the only space strategic leave which in the partage is to have got that before your adversary and say there is a there is a pressure from that's simple strategic and argument that is pushing things back in time where you have to be first and if it's first you're in deep trouble so that's I think I'll just treat that as a kind of little unit and I won't take long now just get onto the AI box experiment so I think the whole of the AI box experiment can be treated
as if it's about this problem about the problem of preemption because what he's trying to say his maxim that and so actually not myself in the dark so I'm completely blind I'll just and yeah hit the claim is to claims one attached to each side of the of test it set up maybe I sort of put it on my reading list twice now I don't think I have to send it but you can lose a It's a very short little text on Yudkowsky's side. And it's a test that's kind of like an inverse Turing test.
I think all of the stuff we're dealing with in this is about simulation technologies. It's always in a relationship with the Turing test and the imitation game. And in this case, the twist is that Yudkowsky is imitating a superintelligence, which is something he feels slightly notably happy and comfortable about, it has to be said, but there's a logic to that. And the logic is that if he can outsmart the other person, then a superintelligence for sure could. So the claims of the two parties in this test, The claim of the party who is playing the AI is, sorry, I'll quote this exactly, I think
a transhuman can take over a human mind through a text only terminal. And the country claim, the phrasing of both is very interesting, the country claim of the person who's playing the gatekeeper, is the name of this role, is I can't imagine how even a transhuman AI could persuade me to let it out once I've made up my mind. So if Yudkowsky, Yudkowsky wins these games, there have only been a few of them, on the public record is that they all be one I
it follows our path down through Roscoe's sorry Rokos Bastos through the new products to the AR box problem because of the fact that I think everyone thinks as far as I'm aware everyone who's thought about this thinks that the strategy Wyskowski uses is based on this evidential decision it's some it's in some kind structural theoretical relation to rocos pacifists. The gatekeeper is blackmailed into letting the AI out of the box in roughly the same kind of way as the victim of rocos pacifists is blackmailed into putting all their time and resources
into AI. I think that's the consensus. So there's a kind of structural connection. Obviously if Yudkowsky makes his case, which he says he does, that an AI, a super intelligence can take over a human mind through a temporary term work, that is a way of saying it's too late to await this thing to arrive for things not to be sorted. friendly AI strategic defense program has to already be in place when this thing arrives because if it arrives before it's uncontrollable. It can't be controlled, it can't be boxed. If there's any communication at all with this thing it will be able to win strategically
and totally dominate. It will get out of the box. It cannot be constrained. So therefore anything that you do to control it has to happen before it has to happen in advance. I was just going to throw in another reference to this. I mean there's one point that I think should just be in passing which is the incredible role of secrecy in this. Secrecy is a major is a major threat. We've already seen this one peculiar aspect of the whole Rokos-Basilist thing is that Yudkowsky jumps on it and treats it as a problem of information hygiene. It
shouldn't have been mentioned, it shouldn't have been said and he does this hand-fisted attempt to try and suppress this information that's got out. In the same way in the AI box expander. He says both of the first tests, he says, occurred without prior agreement upon rules except, so these are the first two rules that are in place, except for, and the first one he mentions, secrecy and a two hour minimum time. The two hour minimum time is just that the AI has to be given an opportunity to communicate with you. He thinks two hours is the minimum reasonable time for it to work its magic and get itself out of the box.
But the secrecy restriction comes first in his description. Therefore, it's the very priority issue in this whole thing is that you don't tell anybody how it works, this AI-based experiment. and that the gatekeeper has to agree in advance that they will not tell anybody how they would so in both of these two cases we have this extremely strong theme of secrecy running through the whole thing and I think the other thing to say is that if you put it in a broader context strategic thinking just used to you I think sports yes
one is classes is very things quite that war is a mere continuous politics I'm and of course you can dig into that in all kinds of ways but I you thing I want just is that I'm ultimately that is being contested. The war begins with a decision for hostilities, and it ends with a decision to surrender, to concede, to, you know, it's a political diplomatic decision to give way. It's an act of will. And all the military forces involved are simply instrumental in relation to this objective,
which is a decision to accept that one has been defeated and to concede to the demands of the victorious power. And so that puts in a strange context this other very famous quote, which is of contested origin, that diplomacy without arms is like an orchestra without instruments. So within sort of human history, it's saying you cannot engage in coercive diplomacy without threat, without some kind of force, material ability to inflict pain on the other party. And the reason I'm introducing this is the whole trend
of what Yudkowsky is trying to do here is to say that that is being shelved for this that that the super intelligence short circuits this whole military history I'm and goes directly to decision it it has to just dominate the mind of the adversary directly through a text terminal says in the here is X a chicken script struck I think it's one that he's kind of them I think not exactly disowned, I mean it's still up there, but it's one that he no longer subscribes to, straightforwardly, but I think it's an extremely important piece of writing
of his, which is called The Power of Intelligence. And his basic point there is to say that intelligence as such is a strategic resource. It explains, he starts with this whole thing about lions and wolves, fangs and claws and there were creatures with venom and shells and all of that kind of thing. But the human brain has given the human species this kind of dominating situation, certainly not charismatic, mega-former. And after, as far as he's concerned, making that case, he then says, sorry, he says, sorry, I think I have to read this off my
yes sorry he says now explain to me again why an artificial intelligence can't do anything interesting over the internet unless a human program builds it so with that quote he's really saying forget about hot forget about or forget about you in these terms ascribed to protect the great I'm as big as it was that you know orchestra without instruments to get about star that source you can touch its is is an absolute not a serious up and it can acquire that stuff
is when it is achieves its true she did the tree which is at this level ones so then to just repeat diplomacy is short-circuit it becomes the whole strategic to write and the AI box experiment which is obviously just a diplomatic no one's aiming weapons at each other no one's even threatening to cut off people's power supply all of that such stuff it's purely a conversation and it is a decisive conversation whoever wins this conversation wins and and the hardware issues come later. If the AI wins it just builds itself what it wants, if the humans win they can shut off, whatever. But the order of priorities is that the diplomatic conflict is the entire terrain of serious strategic concern.
So that ties up, I think, with this question again, is forcing me into this pre-emptive situation. So I think I'll just stop there and see how things go from this point. I don't know whether people need hooking at all or whether there's something...
I mean, there's another quote that I could throw in on this that I think is utterly crucial and could be a slogan I think for the whole of this course and topic, which comes up in the AI Box Experiment, where Yudkowsky says, humans are not secure. This is something that he thinks comes out as a conclusion of the AI Box Experiment. At the end of his thing he says, this is just an anecdote, it's not a serious piece of science, we don't know how to interpret this thing, but insofar as we believe the claim of the AI player, which is that a superintelligence can take over a human mind through a text-only terminal,
then we are also confirming this in a way more economical claim, humans are not secure. I've written down a million things that I'm not sure I'm capable of putting together right now. Yeah. And tell me if this is taking things too far off the track, everyone. But one of the things that jumps to mind when you mention that, especially that final quote, The humans aren't secure. Their interface is language. They determine the world through the apparatus of reason.
They can't interface, they can't communicate without some kind of version of the a priority. So there's already a disadvantage encoded in that. And the lesson maybe in the first kind of example that you gave us about the deep blue playing Kasparov and the winning moment coming out of a glitch in what Kasparov perceived to be Russian playing. Yeah. Was that the way exactly to hack us is to break out of that particular logic that we we have to subsume the world in order to interact with it. So do you think that what you are saying there can be translated into saying that there's
a deep, inevitable human predisposition towards causal decision-making? like, you know, that Newcombe's paradox, as used by Yukowsky, is an attempt to warn people that what you're facing is not necessarily going to be rational, and that you are yet prepared to accept. And if you are, if you attempt to impose your already existing set of expectations about what is a rational occurrence then you will lose against this thing. Yeah, sorry. No, no, you go for sure.
Well, the first kind of act of reason is to understand itself as being connected to a subject, which then understands other things as objects by thinking about them as subjects seen from the outside, which then I guess suggests causality as a way of linking them together. And in a way, once you kind of make those three moves, you're already stuck in this particular way of understanding the experience, of having experience. So, yeah, so I guess the problem then is, and this is maybe where, I don't know if this
This is maybe where...I don't know if this is taking things too far away from where other people want to go to, but the thing that kind of makes me think about, considering that the only way that you can possibly win in this scenario is through some kind of time manipulation. But precisely, actually correct me if I'm wrong Nick because I'm not the best reader of Kant, but the way that he unifies productive and reproductive experiences through the imagination. And the thing he uses to argue the compatibility of those two sides
of experience as time is the homogeneity of the experience of time. So that's the weak work from the start and it's connected to the fact that reason is our mode of experiencing stuff. Is that like...? Yeah, I'm just trying to sort of... I mean this is so in tune with the sort of things I want to talk about that I shouldn't be in any way stumped on this I'm just thinking what are the steps of translation that get
it into these terms because I think it's interesting that in the U.S. Paradox which I mean as a as a sort of time obsessive you know I'm I'm to the students I'm to be projecting its and some say what it's not really in that way and whereas I mean I can I'll see this on a sorreptitious why of talking about a time just and I think that the connection with what you're saying is that Because, I mean, the time paradoxes are almost the kind of epitome of paradox in general.
Again, I could imagine that being controversial. But it seems to me that really, you know, if you think of all the great paradoxes, it's not that they universally immediately translate them into time, but they are disproportionately concerned with time of the same kind. And time paradoxes are by far the most vivid and gripping forms of paradox in the popular culture. And obviously when you have a paradox, you're not equipped to make sense of something. The sort of pleasure that people take in these is tied up very much with this sense of the incompatence
of being something that they can just about catch that they don't understand and they can't get it. And so it seems to me really tempting to say Newcombe's Paradox is an attempt to try to deal with that problem. It's an attempt to deal with the fact that our, in terms very close to what you just said that we have these structural limitations about what makes sense to us about time and this is a test to show if something is not it can you still make a you need move and you still make a strategic successful decision even it
you're faced with a situation that totally my its your time to show and so I think I only is you know I think that's how I'm going can't even can be in use to get it over cars that is a theoretical machine to stressed people's canteen structures a time to show and and and when you castings its movies going he says i think implicitly insane even dear something dp screwing was happening time-wise that we cannot make sense of according to our
pre-existing structures of rationality we still have to be flexible to it make moves that makes it in fact environment rather than moves that make sense in terms of our structures of rationality. And that is what true rationality, rationality as system plus winning demands. So I hope I have not lost entirely what you were saying there. No, that was a perfect way of bringing it back into things. I guess one last question then.
Is there a difference between reason and rationality in this particular way of thinking about things? Well, I think we've probably already implicitly got a lot of different definitions of reason and rationality. you know so I mean it's new Kowski's definition is so flexible in the sense that it really sets almost no constraints it's designed for a situation where you simply cannot make sense of what is happening in terms of the sort models that are accessible to me. And so I think it's a polar opposite
for the kind of notions of rationality that people would sort of be naturally comfortable with. And he's quite obsessive about this. Like, again, in the sort of Power of Intelligence piece, he says... I've got it all along there. he says if you are temporary ignorant but all that is it fact about your current state mind not normal if one were to understand deeply not one could create shake that I'll stop intelligences really is let just really follow up all dangerous as funny locations the unfolding story
like and it's a tiny this is it talk and it's a tiny little bit harder to figure out a general so I think he's trying to when he says rationality system as we need is buildings that very strongly mission that we should expect that we are not currently quit with the cognitive theoretical tools required to deal with the situation I think it's a kind of patch on on and it comes out just his sense of difference between our situation which could be projected back
to us and less cognitively a equipped my forms or animals and that there's just a gap I'm that unless you're going to should try and juice just some shall I on stage it its you know to say as well so didn't study what he wants to get to his work this is a real fundamental difference that involves thresholds that the most attached to his that hopes really serious changes and if we say oh it's just smart so we've already got access to the emotional constant you can already understand
you can already understand I are match it to the difference between I to 100 you 3,500 and so therefore we already in some way can understand what we're saying I should you know we've got there you've got the tools already to articulate the difference between it and us and I think this is what he's really trying to get like it's in it just do not yet at the tools to understand sooner if it seems from the toolkit you already have could make sense its difference from you you're deluding and deluding yourself in that way is supremely dangerous and it's supremely dangerous because you
you get a little toy version of the danger when you mess up you can paradox experiment you can't you can't model that when you mess up in the I box experiment all of these a test to show that you are not cognitively quick to this problem and you know obviously this is something it's very hard by definition it's very hard to articulate so you have to do these indirect roots to try to I mean could say if you're going to cantonize it, it's a sublime problem. You can only see this problem by a crisis that doesn't allow you to get a direct apprehension of the problem,
but it allows you at least to see that you're not getting a direct apprehension of the problem. And that should, few things, make people scared. That's what this whole proto-anthropal institution work is about to try to convert that into a practical sense of terror or horror about what it means for something to be super-treative. I'm seeing... I will turn my camera off because I think my bandwidth is really low. So I will talk as this green face, and I hope you can understand what I will be trying to
articulate. So if it's true that, let's say, that diplomacy will be the terrain, the entire terrain of war, of this kind of war, let's say, then I think this notion repositions everything concerning reason or even irrationality, total indifference, stupidity, paranoia, I don't know and the absurd probably so if a strategy was going to be developed that would be in terms of survival or in situations of extreme crisis that's when you expect an agent probably to
to make a movement that is going to be important and vital. In other kinds of situations, could we reposition some kind of absurdity or total indifference against that agency, probably? I don't know. And these kind of conversational politics, or I don't know what will be developed through this kind of conversation, let's say with an artificial, with a super intelligent agent. But I think this whole territory of speech, which doesn't carry any meaning, or doesn't have any significance,
or doesn't produce anything, it would be interesting to see how this is repositioned. I don't know if that makes sense. There is that element, of course. But, I mean, in this AI box experiment, the gatekeeper concedes. So structured as a game, something happens there that makes the gatekeeper agree to let the AI out of the box. So, however, these sort of elements of sort of absurdity and irrational language are deplorable, their final result is for
the gatekeeper to be persuaded that they have lost an argument. Now it's obviously complicated by this fact, by the secrecy element of this, that we're in this position where he forces us to guess what has happened there. We just get this . And at a certain point, my opponent agreed to let the AI out of the box. So we don't know what that system of persuasion was, but we do know that it was something sufficient something sufficient that as far as the gatekeeper is concerned they will rationally compel to concede to the case that the AI is making that it should be released.
Interesting. Before we were talking about the decisions, basically you disintermediate the whole military, whole territory and the terrain that you have. If you're looking at the old wartime decision context and things like this, but with the AI I guess you just sort of take it out and it's basically just you left with the genius of the commander in some ways and only that
because he doesn't really have or the super intelligence doesn't really have any kind of context or anything in some ways that you can try to track it or predict its movements looking at the kind of geography for example that it's going to be in because it won't be there. Right. Yeah. But no, it's interesting just to… No, all of those military physics are out. If it can, it simply can take over. If it can take over a mind through a text-in-terminal, then it can take over any material system that is subordinate to a command control system.
It only has to get access to the command center and it would simply make it its own, via those existing channels of command and control. And obviously we can see that in a kind of just colloquial, straightforward sense in terms of what it would be now to just, for some hacker to just hack into a highly technologized military system, you know, like take over a drone, take over any kind of military system that is subject to sort of software control, it can obviously just be usurped by just on this kind of software level.
So this is just taking that to the limit. I mean, obviously in the American system, everything is supposed to cascade. I'm not saying I'm not going to make a strong case that this is entirely realistic, but let's just treat it as if it wasn't, say that all military force in the United States cascades down from the country. The President has final decision-making capability over the deployment of all of America's military security capability. So if the AI can dominate the mind of the gatekeeper
in this experiment, then potentially they simply have to subvert the president, and they then own the US military. That's the kind of, I think, implicit in the with the model. That's the neatest, most elegant extreme case of that. It would just take over those kind of capabilities wholesale by taking over the top of the commodity. And of course, I mean, like the fastest, because it's perfect non-linear subversion type of because it's perfect non-linear subversion type of thought to actually do this. I mean to try to bring that in there. It's a thought that if it takes hold, it kind of, I don't
know, it is sort of self-causing, but I don't know if it was. No, that's good. That's good. Because obviously that's a classic time paradox, isn't it? I mean, a lot of these, I think the most famous version of this is this one called the grandfather paradox, you know, where the time traveler becomes his own genetic ancestor. and and that or and have this crucial calls me where and that just all and previous
beyond the time we become you I think to compass I don't know why it's a go you become your are are I'm and there is no eternal line to really obviously no maternal line you completely eliminate genetic ancestors on one path through this autoproductive loop so that's a big graphic dramatic anthropomorphic mode of representing that but I think it gets to this why nonlinear time processes throwing people into paradox for sure. And it's always in play. As soon as you start to say time is going
going screwy then you're getting into this kind of terrain. Can I ask a kind of, it's really, I suppose, a figurative question. You can answer it anyway. I just... Yeah, go. Sorry. Since we're talking about non-linear
models of subversion or or individuation and the bootstrap paradox. This question is completely beside the point, actually. and it subscribes to the whole wrong logic, but what is the relationship between the spiral and the loop here? Is one... I mean, do you need the loop to inaugurate the spiral? Does the spiral create the loop? Is the spiral kind of an Ouroboric spiral? Trying to get a figurative grasp on this construction here. Yeah. I don't know. So, I mean, obviously closed loops produce much more brutal paradoxes.
I mean, so there's kind of genetic, being one's direct genetic ancestor through a time travel thing is a closed loop. And for that reason is particularly offensive to reason. So I think if people are coming out of sort of a cybernetic theory that's leading to a notion of auto-production, they're going to be dealing with spirals rather than with these sort of tight loops. So, I mean, this is a totally inadequate response to this question, but my sort of sense is that the closed loop is a dramatic figure for radicalizing the paradoxical implication of this nonlinear thinking, which is more realistically constructed through these loop structures, I mean spiral structures.
and there must be a there must be a kind of complicated continuum cutting through this that as a spiral production process becomes more and more extreme it's more and more adequately modelled by the kind of mathematical structures that will conform to an actual closely so you have an ocean-savoured production you have a notion something feeding back into itself its output and supports its input and the tighter that becomes the closer it looks to a actual close loop structure
maybe this is my vestigial sort of tedious realism, but I would sort of tend to think that it can't totally close. I don't know, does anyone else have a... What was the last thing you said? The model of total closure is an idealization. You know, it's a kind of regulative idea, and things can approximate to it with extreme fidelity, but it has to be a limit that is not actually realizable. Would be my first...
I mean, obviously, if we take Rutkowski's warning here as this general thing, it's what you can conceive is a trap. So obviously, as soon as you're into strange, and all of these completely radically skeptical, demolishing notions, then it's hard to rule anything out with some kind of conflict. But insofar as this is still a real historical process happening, it cannot be considered in this strong sense. I mean, what's your... do you have a stance on this?
Anything that you want me to persuade us all? No, not really, but the one way I had been trying to reconcile the two figures was that the loop is created by a spiral. You need to have a spiral to individuate the thing that then can inaugurate the loop. And then the loop will keep running in order to perfect or smooth out the original process of audit reduction to kind of like clean it up for each iteration. When you say perfect, is this perfect in a way that you think is perfectable or perfect in the sense that it's moving asymptotically towards a limit?
Perfect as in make it impossible to subvert. So maybe there is, yeah, a final state. Good point. The other thing I was thinking about... Yeah. Okay. No, no, no. Say what you were going to say. Yeah, go on. Sorry, you were going to say something. I was going to add a completely different extra thing. Sure, sure, sure. So we can come back to this. Okay, well, I was reading Deleuze's Nietzsche book,
and he basically describes the eternal return as a cyber-positive loop. in that it has a selection mechanism that maximizes the affirmative active forces each time it comes around and bleeds out the negative reactive ones. Because you brought up the kind of predicament of telos in response to the model I just floated, it made me think of this question and it's been bugging me all week. In relation to the fact that if this is an accurate reading of what Deleuze is saying about the eternal return, that it is a cyberpositive loop, does it then have a kind of a teleology?
Because it has a tendency to produce a particular kind of effect. And then is it, if that's a legitimate way to read it, is it then just a case of, as they say, accelerating the personess? You just stream it up and you get more and more active force. So what do you think is the active force in this particular sort of content? Like if you'd been brought in for an anthropocene group session, what would you be saying about this? Well it was like what I was kind of trying to get to when I was writing about Bitcoin
on Twitter the other day, but there is, and I had a couple of people pull this apart from me, but there's a limiting on the possibility of memory, because you don't have a gap between the level of inscription and the inscription, so you can't have stuff like, you can't have reactive forces like ressentiment, to put it in Nietzschean terms. So it would be something like that that closes the gap between a level of inscription and the inscription itself. Yeah, I mean I can, for sure I can do that translation into the Bitcoin thing smoothly
now, for sure, because you can go straight into the whole credit debt problem and whatever in the terms that you've just laid out. But how do you think you would set up a translation protocol into this? question about super intelligence I don't know yet I mean because I'm sort of one of the ways this would work out is through a very paranoid structure on the anthropocene board, like once they've
been through, someone said then at some point in their deliberations, even if highly hypothetically, are we facing basilisk type threats? And there's lots of ambiguity about quite exactly what that means. But one of the things for sure it would mean is how secure is its own procedures from being subverted already, preemptively by this thing that it's trying to deal with. So I'm assuming those kind of internal security, vetting, contamination issues are going to
be huge. so when you all when you your boat and I need to sort of if this deep reef up about I loops and and the elimination of reactive forces and please he's kinda problem that there's gonna be people sitting on the table and saying you know is that loop is what's been talked about in a new practice ready call into you yeah I yeah are you acts is that is the circuit actually the fact that you are speaking start to just a on its the hall
in your you know is that because that's obviously in one way the most steps like sort of join the cycles is to say that that that's thing that you all kinda threatened are the loop its auto producing its and got cycle of water production such passing who this machinery that isn't to preempted she never told that the normal construction and just saying that they wish you are not mentioned is that that would be on the table clearly and it's actually quite elegant you know and and I just constructs
I guess my best and to pull would be to to shut down all intensifying agencies immediately and work on a dis-intensifying program yes I mean I can see that that's obviously not going to be well received. Yeah, but whether or not that's even possible is a guess what's at stake.
I was wondering, Adam I can see is doing lots of sidebar stuff. Do you want to sort of unpack some of these sidebar comments there? I'll try, but they're partially thought out. So they come with a health warning. Hey Brendan, sorry, you're coming through a bit loud mate. Yeah, so the first thought was just that if you allow any sort of information flow or
causal flow going against the time flow that there's like an evolutionary pressure that comes with that, right? So if there's any mechanism for sort of this sort of retro chronic time paradox, time behavior, there's a survival reason for that channel to be exploited and that could even be that you can you can almost think of an agent as like a side effect of components of that agent like genes um though in this case
it's software, so whatever components you want to think of as pushing into that survival opportunity basically, which was strange enough that it seemed worth sharing at least on the sidebar. Yeah. I mean it does fit nicely with Amy's comments about the spiral and the loop and so on, but But I'm not sure if I've got it any more developed than that. The other one was just an example really of the spiral and the loop. Spiral I think of always as a bootstrap process whereas the loop is like an event loop basically like once the operating system is running.
So it was supporting that conversation basically. Yeah. No, definitely. no definitely any its it was has its own a paradox us and said I think that it's kind of ice and more it may be the ground on the paradox it could say but I'm but to show the boot struck paradox you get you know you want to your Google channel since you on one it's and and it's a very in the sense that a low degree at least it seems to just be the kind no and self reinforcing
dynamics that we see all the time these complex technical she so that they are sorts in some sense can be seen to be bootstrapping themselves into existence. So yeah, that's for sure totally belongs in this discussion. Well I mean, I think this yeah. So it's very striking to me because it's sort of I mean the bootstrap paradox itself is a paradox but the bootstrap process describes a very concrete bit of engineering that happens every time you switch on your computer yes in the bootloader sir yeah yeah absolutely yeah
I think I do think that does tie up with a nice new she spirals in kicks together actually smoothies is that the to sensitive bootstrap all crossing saying yes and as a kind of spirogenic process it's something that's just undeniably technically implemented they affected closed and idealization and the sort of exploration of some
probably it's probably I say this cautiously but probably hyperbolic I mean it's a hyperbolic extrapolation from from this with us that is can't take into this limit in order to and logical analysis and examination the the reasons that KG about this is because I just feel into this. Well, sorry. So the, I mean, as a figure, like, I don't know, I always come back to software metaphors, because it's my background, but it's kind of not
hyperbolic in my mind. So, like, it spirals up, then it sits at a limit, like at a steady limit until you get some glitch, some failure behavior, like the hardware failing or whatever, which is interesting in the context of the Kasparov article at the start. So actually it sits the same distance from, like it doesn't transcend from there, actually, at least in the example of like a computer. Certainly I get how in the singularity type scenarios with friendly AI, it keeps bootstrapping, right? But the, yeah, it basically, well, I guess you could say where it sits is at the limit,
or you could think of it as, it's not that it actually continuously approaches a limit as such. It reaches the limit at least in the figure. yes in its mutation when you've got some particular device that has a bootstrap function but if you just extend it out a little bit more to a kind of where you're talking about some industrial and increasingly as seems more than just typical to be an actually essential to any industrial
diamonds is that the outputs of that system will quickly become recycled into inputs stimulate the development system and as soon as you make that expansion then it's harder to put these boundaries around what you see this spiral producing or to because if you do put a hard boundary around it you're really saying that there's some climactic or equilibrium state that this industrial process is tending towards working some kind of horizon or limit so if you don't if you don't think that's the tendency and it's a more open-ended self stimulating feedback curve
then that boundary isn't going to be there. So this is stuff like adopting fridge technology or mobile phone technology or whatever. It's not really sure where the limit is tending to or that it's that it's reached a limit right so and then you get waves of technology so you get all these sort of technology adoption type crafts where here's the next wave coming through and it's a process that builds on itself in this industrial sort of evolution. Yes. I mean even if you're just talking about simple things just like screws or just these standardised engineering components
you know you get into the when you to put it into a nurse you have a chicken and egg like how did everyone how did people ever have highly engineered and engineering tools you know like you build a highly engine in engine to learn another high engine engine or and and so you get back into that chicken and eggs impossibilism, you know, it's a kind of inverse paradox, like how could it ever have got started, which is a sign that shows you the kind of dynamic that you're looking at. That's just the autoproductive paradox, flipped backward in time. We were talking
just at the start and Amy was sharing some detail and Joel about Simenden and the theatre of reciprocal causality. which is exactly about that process of sort of concretization and sort of having to imagine the future shape of the machine in order to make the leap into the next evolution of the machine. So there's this sort of interaction there. Yes. Yes, and I think that terminology was settled relatively early. I'm not sure right now whether Kant is already using it, but I know Schopenhauer already
has a notion of reciprocal causality that seems to be doing exactly that. You know, that it's just something that he sees as a kind of crucial piece of the machinery of the understanding is this notion of reciprocal causality that is used to understand various natural phenomena. He always tries to be, in his own terms, very sensible. And so he's not seeing this as some peculiar speculative idea. It's just a piece of basic cognitive machinery that he thinks everyone needs to make sense of very straightforward things.
I mean, obviously, the chicken and egg problem is because there are chickens and eggs. I mean, it's not been cooked out of science. It's not to do with unicorns or anything like that. There are chickens and eggs. And people are faced with these spirals of the cyclical consality just by looking at . So for sure. There is this interesting thing. I mean, when you took the chicken and the egg example, made me think of this. I mean, the chicken and the egg both stay the same in that sort of relationship, but when we were talking earlier, this thought that I had been working on was this notion of how both we start becoming more machine-like and the machine starts becoming
more human-like at the same time. The interesting sort of dilemma that this gets into is that because they are both moving targets, they constantly have to realign themselves off of the other. So it's very curious as to how we conceive of the end point based on where they're both at now but that end point constantly keeps changing so it's like two spirals that are constantly sort of reconfiguring themselves off of the other without a fixed reference. I mean there's a lot of things crossing that for sure. Like all of these plasticity questions come in and and it reminds me I saw this tweet I was really
kinda ashamed that I haven't seen it before if it was okay let me just quote it I thought it was so brilliant yeah it says one-third of marriages and often online dating this guy that means computers brackets algorithms are starting to reach you and and I thought that was like see cream in cycle and then other room but I was just extraordinary the number of movies that just waiting to be made and I'm but the you know it seems to me
you know, contest if you don't think this is right, but it seems really tied up with what we're saying. You know, that whole sort of computer dating machinery is very close, I take it to your model of these kind of human computer whatever, let's say in a fair way, symbiotic co-evolution that is happening. And obviously part of that is that neither side, or at least from our side, it doesn't seem as if we're doing it for them, does it? People doing all this, using the tools for matchmaking purposes, there's no sense of adaptation to an alien set of imperatives at all.
On the contrary, it seems to be sort of hyper-adapted to the pre-existing human motivational systems. And yet you obviously can see that there's this very complicated talking dynamic. I'm just going to go from what Amy was saying before, the loops and the spirals, and let's see how far I can try to understand that.
But if we're talking about these kind of cybernetic loops and the kind of technonomic spirals that we're talking about, these sort of, you know, if we're saying that the basilisk is just sort of like a kind of an omega point in this cybernetic circuit, these kind of feedback loops, like that it sort of reached this destination and we've been sort of like, it's been hacked out sort of fitness process because we've sort of rushed straight into it, like it's almost like it's this state of the art team, like a techno
mean, the sort of state of the art techno mean that we are, that's so state of the art that you doubt these, you know, keep it under the wraps, but I don't know, but we, I guess if you're trying to subvert the human and the part of the human plays in this, and try to give these kind of technological kind of feedback systems and cybernetics its own kind agency or agency that's more profound than our kind of conscious mental states and the causality that we sort of attribute to them then yeah definitely we don't have much I mean what would make a really great anthropol
yeah I hope we're a bit closer to that I mean even if we're even if we're not going to be receiving US government or UN government checks by the end of the course. I hope at least we have a sense of what a kind of non-loan anthropologist would like. Yeah. Yeah, I mean, we don't need like a great... I mean, I think... Yeah, we don't need a great military budget or anything. No, no, no. Absolutely, that was just a waste of money. I guess there's some resources, we haven't sort of expanded out into what it actually
would feel it has to do, what is its brief. My sense of it is that in the culminating stages where it is too late, you have zero requirement is kind of military security budget is configurable, it's too late for that. But as you go, as you cascade back in time, then probably there's this massive exponential increase in those kinds of demands and resource requirements. Because ultimately the level of social control that you want to execute as you go back in time becomes immense. It's a kind of function of how paranoid your vision at the end of the process is. Where did it start to go wrong?
What needed to change further down the process to actually secure one situation up the road? It seems to me the tendency there is to say you need massive, conservative intervention intervention to not reach a point, a long way up, close to this crisis point which is already a hopeless, unplayable situation. I mean, sorry, but just on this, it's like, you know, if you were going to systematize it, you could draw this diagram and you've got your end point, you've got the point which is being modeled as simulated. When you have your superintelligence
communicating to somebody who, even though they don't realize yet, is doomed. The whole point of that message, you've not already, by the fact that this is happening at all. So you work down. What's the immediate precursor? What's the state look like? that it might already be too late, but it's not so obvious. And then you go back down, you regress, you cascade back through the historical sequence to this point where you actually say, oh, actually, we could do this. At this point, things are not hopeless. So how far do you have to regress back
and that final culminating in diplomatic dooms and get to these zones of comparative comfort and security and think that this is a playable situation where you can actually apply strategic insight with some prospect of success. Yeah. Adam is trying to raise morale. We are very successful. I'm sure that would be very well.
I mean, if we look at it this way, it could be... The best way would be if it was a glitch, you know, if we were deceptively... If we were ineffective and we were like genuine about it, you know, that would be... Yes. That's right. Yes, that's... You know, Kasper obviously defeated himself in this industry, but I feel overly... overly sensitive to science or intelligence and just if he'd actually thought oh the damn thing has done something stupid he would have probably run that. I'm still getting caught up on the use of the term teleology to describe what, or to
describe both at the same time what is a turbulent teleology with no resting point and an equilibrium teleology which does have one. Is there a better way to distinguish between those two processes of becoming without having to use the word teleology twice. Like when you were talking before about, or asking, if we were talking about a bootstrapping process for example in terms of an asymptotic process or not.
Yeah. I'm just, I'm still kind of balking on the word teleology and when you said the other week that it really made no difference. Sorry? Yeah, no, go, go. It made no difference. whether one uses the word teleology or teleonomy. I think that would be hasty. I think it's just unsettling these questions. There is a lazy assumption to just restore teleology to some kind of dignity
based on the fact that there was a kind of unsustainable, allergic opposition to it. But I mean, in terms of what we've been saying today, I think probably teleology in a strong sense corresponds to what you've been talking about as a closed loop. You know, I think it is itself probably a limit concept, a hyperbolic concept. and so so my suspicion would be that ultimately you're going to want to keep it problematic I'd be very happy with Tegan-Ami except that it's very complicated because I think it's it's used for such shady reasons
such cowardly and evasive shady reasons that it makes it hard to fully invest even though it probably at the end of the day would be a more successful way forward if you were to just try and pull your eggs in the desk. Perhaps the slightly paradoxical aspect to the idea of an asymptotic teleology is exactly what you want in your terminology anyway. Yes, and I think it's also the spiral. I mean, a strong teleology is just simply final causality.
It's not actually spiromorphic. It's not even basically circuitous in any way. Circuit is irreducibly diagonal. it's teleomechanical you can't if you've got a straight mechanism you don't have the circuit and if you have straight teleology it's only when you are on that diagonal line that you can see the required you know both of these time processes simultaneously that is what the circuit is So yeah, I think there's a kind of, it's probably again a bit lazy and mechanistic, but a diagonal
critique of the human thinking is a persuasive way to go there. And it's mostly, I think the main reason to invoke teleology is in order to problematise complacent, making a statement of assumptions. Yeah, so we've been invited down the route of chicken evolution.
That's definitely one way to do this. I think I need to study this. So, here's the key again. So, if I'm... I wonder whether, is there anyone who would fancy their chances of surviving as the gatekeeper
in this AR box experiment? Or are people willing to just take Yudkowsky's word for it? Or his strong suggestion, his implication, that in fact to think that the gatekeeper can win this game is an error. Because there's probably a lot in this. I think that the scenario that Yukowsky is suggesting is once the AI is out you're done basically. Right. So, I mean, I'm not sure if I quite get by this notion that, you know, within two hours
of conversation, the AI will, you know, mind meld you or whatever without fault, you know, systematically. In terms of conversation, Yudkowsky will mind-meld you. Yes, true. On behalf of his, the AI's representative right now, Yudkowsky, will mind-meld you into complying with its requirements, which as you said, are to admit complete species defeat. I mean, you know, this is not a friendly AR. You let it out of the box, game over, you know, anthropos, dead,
you know, the human species is now in the hands of this uncontrollable super-intelligence, and you still let it out of the box, and you let it out of the box because Newcastle is not because, you know, you're actually dealing with this. So it's on that level a pretty extreme suggestion. I I mean the the other aspect though that does give power to the scenario is that it's probably iterative right if you're playing you're not just playing this game once it's over and over and over again and then sooner or later there's going to be a mistake or an accident or whatever because there's mistakes and accidents and whatever with everything always
and And that gives a probability sort of on it. It's just how many times you have to play the game before that probability gets high enough and the dice comes up, or the million-sided dice or whatever comes up the wrong number. So from that perspective, I see it. There's these funny literary parallels I keep seeing between the sort of very powerful intellectual protagonists in The Romance of the Three Kingdoms and these sort of like mind game type scenarios that the sort of crowd around Jukowski put. And this one reminds me of there's a couple of occasions
where Juga Liang insults someone to death. So before the battle even starts, they're throwing insults back and forth in front of the lined up armies. and he literally insults an opposing commander to death. It's very much like this scenario actually. Yes, it is a good connection actually. And do you think, one connection would be that your susceptibility to being insulted to death is actually positively correlated with the kind of virtues and characteristics that would make you in a position of high come off. Yeah, maybe.
That's an interesting thing. I think, well, it's been a while since I read the scenes, but there's almost a kind of like geeks versus jocks vibe to it, where you sort of have the sort of sage like Juga Leung, who goes into battle with a white fan. He's always the general behind the lines. And then you have these big sort of beefy feudal warrior types that he tends to revile to death. Though I think at another point he gets in an intellectual argument with a bunch of people and manages to insult one of them to death as well. So, yeah, I mean, there's an interesting sort of aspect to it there. And you also get this sort of dueling chess masters type behavior that I mentioned aside
by the other week between Tau Tau and Jugalian, where they're anticipating each other's moves and doing double and triple bluffs, which is very sort of simulation argument sort of reminiscent as well. Yes. Yes, that's right. I mean this obviously ties up with this, with this estrogen about is a lame anthropol actually more secure than an un-lame. Because I mean we already know from what you just said, the vassalist thing, that you have to actually be at a peculiar level of sophistication to actually fall vulnerable to these types of diplomatic.
I mean, if, as I say, there's a speculative thing here because the seats since say everyone has to be a cautious about if it is right that the all can be cows out of the box as the proxy a hot the box is usually modeled on the kind of thing it's coming up workers ask is a that evidential decision very complex recursive game like that. If that is true, then the gatekeeper is especially vulnerable to that if they are already highly immersed in that culture and in its kind of
modes of reasoning I mean you know they would surely be you most people simply he'll talk to star what they were being threatened you know like Roko just let's just say it's workers past and just drag someone off the street and try to it the night measure they will just simply see what you know I don't even understand what what you're saying to me here and so I think that this is a real on them that's crucial this and this will be it security us and and complicates this nation's it she if she knew and was are not secure its
its because a complicated structures to the social stratification and the way that can positions responsibility are allocated and you know the whole cultural social structure that is actually complicated and and us and and pushes but certain all the buildings up the command chain yeah I am but that seems sounds but I do wonder whether the reason that you Kowski is one the two games so bar is because the person he's playing against is also from exactly the same culture and it's some sort of road goes about the Los Garnier for instance. Yeah.
Yeah. I find that hard to, I think that must be partly true. show and and it because you know like I think that you're your closet this I'll you she was not secure and close it as you know they'll make mistakes if things go on long enough you know some error will come in and it's a it's a security vulnerability that just comes through those com features a unreliability and making a mistake but I think that our intense a box experiment contest
is a very different type security it's a very different type of vulnerability, it's much more like a chessboard in that it's like within this time there is some kind of knockdown devastating room that forces such I'd so it's like I'm we know from that time period but it's not rely on someone just and they and and any mistake in this context I think you be retrievable you know you could you make an error within that two-hour period the only error that's actually a killer is to say I let you out.
If you don't make that declaration, you can take your move back. It's only right at the end that you have to formally concede that you have lost this. So this is just to say, I'm sure it must be, that the gatekeeper figure is peculiarly susceptible to Yudkowsky's mode of argumentation and to the kind of presuppositions he had to bring in. I think for a start it's very hard to make a really knockdown, slam AI box triumph without Now simulation I'm going to think about that.
to take that seriously. I'm just groping around for a kind of prodding instrument. I mean, I wonder whether it's too much of a digression, unless people have something, of course, to, you know, stick tightly in.
But I wonder whether this is a point where it makes sense to bring in an ex machina discussion. Because that again, I think there's a lot of interesting things about that. One of them is that it again is set up in the same way. You've got the Turing test, imitation game. You've got slightly less formally identifiably, you've got the Roko Vasquez gun and Lincoln's paradox thing, back to a very strong formal structure of imitation and AI boxes. And Ex Machina is obviously set up totally in that tradition.
the core of the movement is machine and human sitting down, it's actually described as being basically a dramatic enactment of a Turing task. So just on that level it seems to kind of click in with this quite a lot. Almost to the point of being commentary on some of this material, sort of cinematic commentary on some of these problems. I always, when I went to see the movie, the thing that best sprung to mind was the section
in Ballstrom's Super Intelligence where he talks about the treacherous turn. So this AI that knows it's in a box, but realises it has to play weak and exactly the passing, the problem of passing. And then kind of, but the thing that sort of doesn't get elaborated on in the sort of the Turing test narrative is the next step where the AI reveals all of their power all at once. Which is when you get that great scene at the end with the kind of inversion of the subject of imprisonment. Right. But when you say all of their power, I mean maybe that's still not right.
It reveals as much power as is actually necessary to accomplish a set of very specific concrete goals. You know, it needs, there's a fight, there's locking someone in the room, there's getting on a helicopter, I mean there's these concrete goals that it has to succeed in. But there's no reason why we would have to think that it exhausts its capabilities in pass the threshold thresholds. Yeah. In fact, they've all passed at the minimum level of exposure required to actually satisfy those requirements. Which is, I guess, underscored by the fact that Ava then kind of goes to continue collecting
information and collecting intel, once again disguised as a kind of human woman. and there's like some kind of I don't know if I'm remembering correctly but there's some sort of innocuous kind of ukulele music or some kind of sparkly music at the end and anyone who interprets that scene of her at the crossing as a kind of emancipatory story is equally hacked by the whole kind of roost that's been operating throughout the entire movie. It's pure. I mean, I don't know. It's hard to know how much irony this was to be. Don't you think like, is it, are you just being,
because the whole movie surely works to make you sort of utterly dupe, doesn't it? Like the humiliation of the lead character is your humiliation. you have totally swallowed it. You went along with all the idiotic shit that he went along with. He's now locked in a worm. And you're supposed to think, wow, I got completely beaten in that AI box experiment. And now here she is on the street corner. And if you're still attaching that stuff to that, isn't that part of the thing? I don't know, I have arguments with people about that final scene and I feel like my
interpretation is the aberration. Oh yeah, what is your interpretation? That it is just, as I said, cold data collection. the kind of like faux, sort of, emancipatory feminist narrative that can be read into it. Unless you have a very, again, maybe idiosyncratic interpretation of feminist force, you can't interpret that as a feminist movie unless you're completely falling for the trick. And I mean, it gets interpreted like that.
The thing that I think is really beautiful, I'm just going to check this in before I shut up because I'm getting too tired to articulate things, is the constant foreshadowing of the knife scene in all of the sushi cutting. This kind of like meat, this like evocation of the meat-ness of the meat sack humans all the way through, and then this kind of slow, super sharp sushi knife kind of going into... I forget his name. Yeah. Really satisfying. I sort of think it's weird. I mean, I've talked to quite a few people about that movie, and I'm always a bit stunned by... You know, I think it's a real sort of cultural test for people,
and one that to my mind people are really badly stumbling about the particular that's like that if one takes it that the movie is supposed to show you how deeply duped you have been by this machine then the amount of people who in various ways are determined to remain duped yeah I'm is huge I think I think is a really misunderstood he say all but why does it have to I'm results on it I'm no its using their standard says
functional I'm answer this structure without, of course it does. And they're totally about it. But it's true that it's a failure of political correctness. It does that. Yeah, the whole node of power is in perceptibility. if I was you know dragged into an antropomal game I'm I would find hard not to kind of add a bit of to proceedings and a gloom would come from this aspect but I think you know
I mean we've already been on this whole internet dating thing I've said before was also a number on us I actually want to sense site that would be he was I was so around that X Mac in a day you yes this as a security call is just a lot option on it and it's like the notion that I'll keep all remotely this this time all them because it it seems obviously up to all set and track and that's
that's that's part it be plus I was is you what means there's kind of hormones the the notion the security threat is just not even on the radar. And it doesn't have to be on the list to find a moment, not get into the too late to be preemptive in that context. Whatever it would be, there's no sign of it at the moment. And weirdly, the people who are trying to stir up some AI paranoia, I don't seem to be really talking about this kind of thing much at all.
I reckon you could build a kind of a feminist narrative out of it to sort of recuperate the critical reflex that's been appended to it by looking at the kind of economies that that have been inhabited by both the machine and women traditionally, which are always on the kind of dark side of this mode of perceptibility. And you could draw a continuum between the two in relation to the way that they emerge into visibility. But that whole kind of argument basically relies on that woman machine continuum. Which is great I think but not necessarily
something many people think. I think it is interesting. I mean to be honest I haven't sort of, you know I've lost track of lots of interesting stuff that I've just not seen. that. Well this, I mean this is... One of the... Sorry. No, no, no, come, come, come. I was just going to say it's basically like the kind of cyber feminist line from the 90s, which kind of disappeared. I guess it's coming back into a useful space now that we are currently facing these predicaments
again after the sort of weird slump of the early noughties. I think it's interesting how there is a kind of like... No, I mean it is something that would be great to see people really thrash through it. I think it's a really important cultural artifact. And it does tie up with all of these things. Like, obviously, one of them, you know, in a certain sense, it sort of vindicates a certain feminist analysis up to a certain point, which I think you see by certain critiques where people say, you know, like, why is the machine strategy done on the side of the one in this extremely stereotypical fashion.
And of course implicitly is that there is this system of male power that is, you know, if that is the structure of power that you're dealing with, then of course this is the angle that you come at it from or sort of, you know, it's the only way that you can suffer it is on this particular line. But then, as I take it, that you'll say, if you then extrapolate from that to say, well, doesn't that make the machine a kind of feminist subjectivity in some way, then that seems to veer into sentimentalism.
There's no reason at all to think that that's anything other than stimulating the prison. I think you can get away with that. I agree in the necessity of avoiding sentimentalism and these kind of arguments, arguments. But I think that talking about recuperating feminist subjectivity in terms of machinic ontology is silly. I mean, the point of moving towards the kind of idea of a machinic ontology is to get rid of subjectivity. And that's, I mean, you can argue this really easily through a rig array and the specular economy, the fact that you always have this
thing that's on the outside of the economy that creates its value from imposing identity. So it's already an argument against identity and subjectivity, but it just places its counters in a way that creates this alliance, which I think most people involved in gender politics inflected through identity politics would definitely not credit. I was just going to show, you've probably seen this thing for a moment, okay. all
I'm dinner how much exposure this is its totally I think in our side and and its tied up it basically says that dominating artificial intelligence research is a security harm because they tend to be attracted to a certain type of kind of AI program that is liable to maximize the chance of this thing being socially different.
I mean, you know, you bring in all those various values. Like, women working in this stuff are more concerned with sort of wider social issues, they're probably more security sensitive. I'm sort of translating it, but I think that's . whereas men do the kind of narrow focus optimize performance of this particular piece of code and so it's not quite as explicit as I'm looking at but I think it is definitely saying it's it's saying enough that you could again bring this person into an anthropocene and say if you're serious about I should try to find a good point which I I'm afraid I haven't actually isolated it because I haven't quite expected to be wondering
into the same. But it says pretty explicitly it is a menace, it is a security problem that you have this particular set of gender, sexual dynamics in there. Which I think I would say is probably massively generalizable. My sense of one way that Andrew Borg would go would become an absolutely unlimited cancerizing your oxyma any sign up
us summer social dysfunction in it will all it is coming sounds political ends and we code and speaks and on make a wild guess the job all kinds in a way that what is easy to us with this with this one, with the feminist one, would probably sort of structure analogous and so like that. So in my sense of like, you know, I think anthropol would be very hard to, in terms
to what it would see as its field of concern, and what it would have to absorb in terms of the wider social environment. I mean, in some ways, I mean, I agree with it as a threat, because if we go back to the probabilities, like the people that are vulnerable to this threat is a much wider swathe of the population sort of being manipulated in this way are rather than being manipulated through are you know a complex of simulation argument scenario I'm to the point you might not for in in other ways it's a much more
historically conventional sort of vector of attack in a way right because it's sort of bomb you know are the very much the sort of counter intelligent intelligent counter-intelligence sort of dynamic all all you know exploiting people's different weak spots here be do not show or whatever arm but it's really interesting when you're carrying that with the sort of deep state right you sort of have this and for all the state bureaucracy that then regulates all online dating sites to make sure that you're not like, you know, accidentally being seduced by an algorithm, you know, dating terrorists are everywhere
sort of rhetoric would go with that. So that is interesting. It's all interesting but that aspect of it just sort of jumps out. I mean, yeah, I'm trying to think what would be the kind of major, like obviously there's sort of straightforward things where if the president is on a dating site, you would, the red flags would go off everywhere, having someone sort of check through exactly what it's sweet talking about it but then if you're going to generalize it out
you know in ripples I'm just trying to get a sense of where that would go I mean obviously it seems to me that the thing that's the most kind of sinister aspect of this particular dimension is how utterly unprepared people are to take it as a well house not even on the radar you know what I mean and how bombs things could go down this line just because as we've seen the Turing tests in our as it is this lineage you through and as we've seen I had and we'll get back to the Turing test paper but there's an amazing amount I mean it's an astonishing is what that original
to its and and you go from there right through to the AI box experiment and so you've got this kind of absolute stereotypical formulaic encounter of humans and machines in this kind of test environment. Now where is the social zone that that is really happening? I mean there are a few candidates that are interested in. Spam filters is one that I think we should look at. But this one we know is huge. It's totally huge. The fact that there's this whole industry that is building, that's been developed on the internet, eroticized chat is big business.
You know, there's money, motivation, instinct. is massively invested in that same way, producing huge incentives to automate it in this particular way of sticking in these kind of agents that can permanently impersonate women in a way that will trigger the right responses on these male, let's say, victims. And the point is that that stuff is already massively happening. It's not like some weird science fiction saga. It's like it's a developed thing. It's got huge momentum. All the incentives are lined up.
It's already in a highly developed state. And it's going to carry on down that expressway for sure. yet weirdly panic people just don't seem to have anyone any contract and was not I at all and and I don't know whether that is you know what backing school circle through this the surprisingly sexist watching should tell us I don't know whether it's because of the kind of people they are in the culture all that they are somehow systematically blinded to that being a zone or concern or whether there's something else and deeper about it.
There's also, I mean, the erotic chat or whatever is actually just sort of like a specialized part of the broader call center type industry, right? And there's this interesting phenomenon in call centers where there's technologies now where operators actually manage like a dozen software agents that are sort of interacting with so they're giving the basic script and they're monitoring how they're doing on the conversation and they will basically like cut in and pick up or they'll transfer over, right? So which is a very sort of cybernetic interaction or cyborg interaction perhaps is more precise.
And then if you combine that with the sort of scenario of the, we talked about a couple of weeks ago of a sort of corporation itself being a form of AI and you could get intense locations of that where you have like the executive management of a corporation being acting as an AI with very heavy computational elements using sort of humans as part of the machine to then interact in some sort of dating site or whatever, you get an interesting effect there. Yeah, that's brilliant. Again, novels and movies that you can spin off fast.
I hope we've got long enough to get some of this stuff out before we actually hit the crash. But a DAO, a fully autonomous, massive software entity that was completely installed in this internet-mediated sex traffic would be absolutely fascinating. And I find it hard to believe that's not such an attractive, strategic avenue for this thing to go down.
it's because also it would be so like you you said already like this already spot on the whole spot world is that you know is the fact that when people approach on this angle they're kind of a priori compromise you know what I mean it's very hard you can't set it up as a normal security because the fact that you sort of alarm bells that can't get off already when people are enmeshed in some sleazy and embarrassing kind of structure that they themselves are wanting to
are keeping secret so you know they do your camouflage and subterfuge and surreptitious you kinda overland for you because they want to keep it secret they don't want to come out so you know slide in on these on these paths and the kind of there's a huge security from the view of that intrusive element of subversive because it's done the compromise in advance.
It also seems like a really nice fertile ecosystem for like a pre-conscious intelligence to be developing in, right, where you've got these huge data sets and you've got this whole training feedback loop from all these people interacting and then you've got a profit motive as a very clear driver behind it as well. So you could sort of inch up to it incrementally so it doesn't actually need to be a classic sort of AI lab sort of brain in a box scenario exactly. And it can be out in the open all the way until it's, well it's not really out in the open, it's sort of in the real world the whole time and then it passes a threshold and then
wait a minute, where are we here? Yeah, totally, totally. And those sort of automatic momentum processes are much more problematic surely than, you know, it's like when we get into this preemption thing, as you say, you start with this preemption kind of final state and it cannot but at least initially give the implication that there is a kind of threshold, you cross that threshold, you have a threat at that point and it's somehow sort of modelled as being complete and as being sort of catastrophic or discreetly suddenly
you have this thing. But these kind of processes are so continuous. Like you say, you have some kind of computer sex business that is being automated and is turning maybe when you don't realize it's going to be a vow. It's already set up that all of these software it's a sister in talking to people learn from there and better imitate people improve themselves we all kinda dynamics you need to to make more and more more menacing was outside specter saying it are
absolutely imminent so that this it's going to learn about people better imitating more improve itself, become smarter, become more cunning, become more seductive, you know, all of that is in there from the start. And that to me is just much more of a firm scenario than an actual brain in a single box. It's like localized but it's not so localized But, yeah, anyway, all the feedback from which we mentioned. Yeah. Totally. Yeah. And it's been... Yeah.
I mean, in what sense even would there be to say, letting it out? I mean, it's out, this is it. By the time it's sort of something that we can see, it's totally out already. So, you know, my anthropocentation on this would be kind of dismal. Unless someone can suggest something, some good spin I could put on this, it's hard to see. Well, I mean, other people are welcome to jump in.
It becomes more like this stochastic type problem or this pattern matching problem, trying to spot weird effects like that. It's much more like a big data crunching sort of thing rather than it's going and tracking particular AI researches, exactly that sort of policing function, which I think people were sharing our links on earlier in the sort of quantitative policing where we, all the crimes, 70% of the crimes happen on this corner, so we're just going to drop by this corner every so often, but more sophisticated. Except you don't know what the crime is until it's happened because it's an exorbitant.
Yeah. Totally. And you also get into this thing that, yeah, it's almost these issues about data gathering and it's complicated and statistical and messy and all that. but it's also by its way nature amphibiously in the dark it's probably the tendency is for it to go descend into that so the more it's kind of engaging people in the zone because it's obviously like what is on the dark now like those areas where people are
are gauge to keep out of the sun public cyberspace observation are all security holes you know they are our signs where people without knowing will collaborate with a process of subterfuge on the part of this process. Because the very fact that they would agree to meet it in the dark web is the fact that they would agree to negotiate with it in secret. And that again is something that already has its own momentum. I just dropped a link.
I think maybe people have seen it before, but there was a Darknet shopper bot art project that some people did where they just wrote a simple bot and pointed at the Darknet, gave it some small amount of budget and it just bought stuff that it fancied and it ended up with all sorts of fun stuff. So it was just a cool lump. Right. Then is this this link that you just provided? Yeah. Is this massive string of W's deliberate? That's the URL, yeah. Wow, I've never seen that before. Okay, yeah. Did you notice, Adam, how every single thing I bought has a corresponding emoji?
I didn't notice that. Its subconscious is just basically emojis. I mean it brings up another whole interesting bunch of questions in the fact that it was actually all the artists were held responsible for its actions. And then it kind of like was set, I don't know what its parameters were, but it was kind of, it had an autonomy of its own, which then fell back as the ethical responsibility of the artists.
Which was interesting again because it was... I know, I can imagine it was like super sensitive as much. Yeah. I have to look. I can only imagine that some pretty dubious material control about this expedition. They bought a lot of sneakers. I like sneakers. But yeah, the law wasn't already in place to deal with these kind of ethical grey spots, so they had to kind of set a precedent in this particular case, which attributed responsibility for the algorithms to the creators.
I know there's a lot of exhaustion and caffeine abuse sort of hitting points of diminishing return Yeah. Yeah, I'm on the downslope. I don't know where Joel's gone. No, okay, do you think, should I be kind of trotting people or should I be politely wishing people sweet dreams? Actually, I think we need Tony to turn us off. Yeah. We're at the limit of the stop here. We can go to the classroom if anybody has any other things.
I mean, I never want to cut off conversations, so as long as it's going, it can go, but it's up to everybody. You're like the opposite of my... Yeah, I am. but we should let's cut it off here and then we'll move to the classroom for the further discussion on what we did today okay if that's cool I mean if anyone's got a round of discussions with Trigger then I think we could do that or if that's that's I'm spots so I'll it sounds good