Anthropol The Future of Human Insecurity (Session 5)

Hey. All right, so thanks for joining us for the fifth session of the Anthem Poll, the future of human security. We'll start with Nick. Nick, are you there? Okay. Yeah, so we're off? Oh, yeah. Yes, we're off. Okay. Okay, so as agreed, I hope, last week, when we're sort of hit peak insanity with Roko's Basilisk, and are now going to systematically work our way back down the slope, revisiting

the topics in reverse sequence and ultimately getting back to the question of intelligence explosion right at the end. So I've got really three little introductory remarks to make this time that I hope will sort of help to get things rolling. First moving towards the AI box experiment from Roko's Basilisk down through Newcombe's paradox to the AI box experiment.

And then talking a little bit about the time structures that are coming into this material. And thirdly, which might be the best thing actually to start with, is a little bit of a digressive element, but I think it's suggestive in a way that might be picked up on and connects with other stars. So one of the things that Yudkowsky takes from Newcombe's paradox is what leads him to his maxim that rationality is systematized winning. I see that in the same way as no

one foreign to geometry enters here above Plato's academy. It's a kind of an anthropole slogan. I'm assuming were anthropole to be set up, its brief would be definitely to win. any definition of rationality that it came up with would be within that context. So I thought there was an extremely interesting thing came up which is to do with the chess match. I'm sorry, I'm just going to post a link for you guys. in case you haven't seen this it's about

um Kasparov's famous test match with Deep Blue the first time that a computer or algorithms could be said to be the world chess champion was by defeating Kasparov in this series of chess matches it was very tightly fought and Kasparov famously said that at a certain point in his matches against the blue he caught an intimation of an alien intelligence in this chess program. Now what is interesting coming out of this new material, there's a little video attached to the

link that I've put, is that it looks now as if I'm deep blue actually glitched out at this crucial point in the match is described as being caught in a loop that the program actually couldn't make a a move that it was kinda I'm content about and ended up just but basically making time-wasting move just in order to break its way out of this this loop it was caught in and and cat the cacus most my man away or he was seeing this alien intelligence corresponded to a moment where this chest program is actually

fritzing out its doing something that doesn't make sense to him and didn't make sense actually to the program and itself and was a move that was kind of random and and so Kasper projects onto this this this intimation of an alien intelligence now there's something I think quite deep about this it's tied up with a lot of other parallel phenomena and but I think just to take away the media take away is just this

a element that can purely chance mistake a glitch actually plays this role but if we go back to the Kowski comment context you can start up is what wins you know deeply wins this chess match if rationality is systematized winning this is what actually counts as the most exalted moment of rationality in the in the deep blue program and it was just a mistake it was it was you know by as far as the programmers were concerned as far as all the kind of rational context and background to this concern this is an error

it's a software failure and it actually produces this winning opportunity uh... so castro is defeated by mistake by software uh... say that's i'm just gonna leave that and little box on its own but it seems to me extremely suggestive in in various ways uh... soon now uh... let me i've just gone to time this site at go on to out this uh... maybe i'll just quickly run through the time stuff because i think that that could become huge and and and we don't want it to necessarily at this

point obviously people can't about it but i think that the point to start with that is in the whole popular call culture surround this question of AI x-risk, there's a huge role played by the Terminator franchise. And I think people almost now have stopped seeing it as a question. Why should it be that within that particular myth, the great popular myth of this menace, that there there is simultaneously the question of the arrival of artificial intelligence and time disturbance. Why is it a franchise that is simultaneously about time travel and about

artificial intelligence? Now one could obviously take that just as a complete random coincidence or two things have just been kind of glued together without any deep motivation there. But I think that we can already see on the path we're taking sort of down the slope from Roko's Basilisk that there are reasons to think that there are connections between these two topics, if they are indeed two topics. And I've sort of subdivided them into three pointers that are all, I think, independent to a certain extent.

And going from, you know, following our downslope path, the most crazy of these connections is obviously what you see with Rokos Basilisk itself. Because Rokos Basilisk notion of acosal trade is the possibility of a negotiation with an entity in the future, as we were saying last week. You can engage in acosal trade with something that is infinitely distant from UN Spatial Target. And obviously, in the context that works with the particular rocose basilisk model which is that this is a future artificial intelligence with which one is being involved then that reduces that collapses this kind of infinite possibility

of distance to something more specific something more like the terminator model that there is a communication taking place from its point of view backwards across time that rocose Newcombe's Basilisk is a mode of retro-chronic incursion from the position of a future artificial intelligent entity. So that obviously is one way in which this gets, I think, quite explicitly set up. The second, a little bit more subtle, is tied up with Newcombe's paradox. And we go, I think, uncontroversially we're going down the slope to the Newcombe's paradox.

Newcombe's paradox is taken seriously much more widely than Rocco's Basilisk. I think that's totally, one can say that with great confidence. And that it's taken seriously, I think, comes from the fact it was publicized by Robert Nozick. That's where it really receives sort of general attention. And he himself comments that everyone thinks the basic problem posed by Newcomb's paradox has a simple solution, but people divide into two roughly equal camps about what that sensible, obvious solution is. it's you know there's a 50-50 split between using

language we we introduced last time between whether you're gonna accept a causal decision theory or an evidential decision whether in the specific terms of Newcomb's Paradox you're going to be a one boxer or two boxer as Nosek says you know people fall into these two different camps and each of them think that the other side is being clearly irrational and so the reason the fact that there's that division fact that there's just total lack of consensus about what a rational strategy is in this situation is I think the reason it gets taken seriously people are forced to take it seriously because there's such a disagreement

how to solve it it can't be neatly shelved by people just settling on one or the other side this time and and I think Newcomer's paradox as a whole can be constructed quite straightforwardly as a time travel simulation it explicitly but the basic problem as you remember I hope is that and Omega as the less wrong people call it or the predictor as it's originally called has either put a million dollars in this closed box or not and at the time that you are actually making the decision

do you open both boxes or one box its closed and it's going to be no other activity taking place in terms of that box at all it's over that's why the causal decision people say look Omega has either put a million bucks in there or not So why, so it can't take it away if we decide to open the other box as well and take the thousand bucks from that. Of course the evidential decision theory will say, but the record shows that every time people do that two boxing, or almost every time they do it, they lose. So by again, the Yudkowsky Maxim rationality system vice winning, you go for one boxing just because that works even if you don't understand why.

So this prediction or prophecy that Omegara is supposed to take place, I think is perfectly isomorphic with a retro chronic action. There's no difference ultimately in the structure of something about perfect prediction and ability to influence the past, an action of the future upon the past can be for later as a prophetic or reliably predictive act because if something from the future about which you are making prediction or prophecy and is confidently held to be going to occur then that thing is operating

through you through the prediction now and this the they can be just converted into each other perfect perfect we got perfect confidence those two things so and new comes parks isn't set up as a time travel story but if you were to set up as one you would not have to extra at all and I'll look at it could you could just set the whole thing up saying and Omega is able retro chronically to insert or remove a million bucks from the closed box. The only reason that you wouldn't put it that way is because that would make the causal

decision theory approach, the two box approach completely and obviously irrational. It wouldn't be a serious question at that point. But if you're a hardcore one-boxer, then there's no difference between your models in those two cases. If you say the reason I'm taking only one box is because I trust Omega's predictive capability and if I take two boxes, I know it will have predicted that and there will not be a million dollars in the closed box. Or if you say Omega has capability and it can, from the future, change whether or not as a million dollars in a box. Those two things are exactly the same once you have taken the one boxer interpretation of the situation. So that's the second way that time disturbance

gets moved into this. And the third most sensible and I think simply undeniable, crucial, we'll come back to it a lot in what we're going to be dealing with, and I think in future weeks, is just preemption. As we'll see, I won't go into this in detail now because we'll be dealing with it in the AI box experiment, but it's absolutely fundamental to Yudkowsky's position that you cannot, his whole system of argument is set up to prove that you cannot deal with this threat in so to speak real time at the point that it's there

at the point where you have created a superintelligence it is too late to deal with it there is no possible strategy that will work we'll see how he tries to argue that and that means that you have to deal with it in advance so it's an absolute fundamental strategic tenant of also answer poll orientation it in so far as you cast has anything to do with setting up but it has to be preemptive it has to deal with this problem it up of it becoming actualized and obviously that tends towards a cascade once you once you are in a sort of a

a a competition for preemptive action there's no natural equilibrium point you always want to get back earlier and earlier if starting from now you want to be faster and faster and faster at addressing the problem but once it slips by another stage you start looking into history and start looking into the past and start actually analyzing things in terms of what has already it perhaps in some deeply suck tissues a camera much way already taken its you know the pressure is always to have started at the earliest possible opportunity and so something that happened already is more dangerous and something but it's gonna happen

in the future the only space strategic leave which in the partage is to have got that before your at history and so there is a there is a pressure from that simple strategic and argument that is pushing things back in time where I'm you have to be first you have to and if it's first you're in deep trouble so that's I think I'll just treat that as a kind of little unit I'm and I won't take a lot not just get onto the AI box expand So I think the whole of the AI box experiment can be treated as if it's about this problem,

about the problem of preemption. Because what he's trying to say, his maxim for that, sorry I've sort of locked myself in the dark so I'm completely blind, I'll just... Yeah, the claim, there's two claims, one attached to each side of the test. It's set up, maybe, I've sort of put it on my reading list twice now, I don't think I have to send it again, but you can, it's a very short little text on Yudkowsky's side. And it's a test that's kind of like an inverse Turing test. I think all of the stuff we're dealing with in this is about simulation technologies.

It's always in a relationship with the Turing test and the imitation game. And in this case, the twist is that Yudkowsky is imitating a superintelligence, which is something he feels slightly notably happy and comfortable about, it has to be said, but there's a logic to that. And the logic is that if he can outsmart the other person, then a superintelligence for sure could. So the claims of the two parties in this test, The claim of the party who is playing the AI is, sorry, I'll quote this exactly, I think a transhuman can take over a human mind through a text-only terminal.

And the country claim, the phrasing of both is very interesting, the country claim of of the person who's playing the gatekeeper, is the name of this role, is I can't imagine how even a transhuman AI could persuade me to let it out once I've made up my mind. So, if Yudkowsky, Yudkowsky wins these games, there have only been a few of them, on the public record is that they all be one I it follows our path down through Roscoe's so workers past this through the products to the air spot them because of the fact

that I think everyone thinks as far as I'm aware everyone has thought about us thinks that the strategy cask uses is based on this evidence its its some it's in some kind structural theoretical relation to rocose fascist. The gatekeeper is blackmailed into letting the AI out of the box in roughly the same kind of way as the victim of rocose fascist is blackmailed into putting all their time and resources into AI. I think that's the consensus. So there's a kind of structural connection.

Obviously if Yudkowsky makes his case, which he says he does, that an AI, a super intelligence can take over a human mind through a temporary term work, that is a way of saying it's too late to await this thing to arrive for things not to be sorted. friendly AI strategic defense program has to already be in place when this thing arrives because if it arrives before it's uncontrollable. It can't be controlled, it can't be boxed. If there's any communication at all with this thing it will be able to win strategically and totally dominate. It will get out of the box. It cannot be constrained. So therefore

anything that you do to control it has to happen before it has to happen in advance. I was just going to throw in another reference to this. I mean there's one point that I think should just be in passing which is the incredible role of secrecy in this. Secrecy is a major is a major threat. We've already seen this one peculiar aspect of the whole Rokos-Basilist thing is that Yudkowsky jumps on it and treats it as a problem of information hygiene. It shouldn't have been mentioned, it shouldn't have been said and he does this hand-fisted attempt to try and suppress this information that's got out. In the same way in the AI

box expander. He says both of the first tests, he says, occurred without prior agreement upon rules except, so these are the first two rules that are in place, except for, and the first one he mentions, secrecy and a two hour minimum time. The two hour minimum time is just that the AI has to be given an opportunity to communicate with you. He thinks two hours is the minimum reasonable time for it to work its magic and get itself out of the box. But the secrecy restriction comes first in his description. Therefore, it's the very priority issue in this whole thing

is that you don't tell anybody how it works, this AI-based experiment. and that the gatekeeper has to agree in advance that they will not tell anybody how they would so in both of these two cases we have this extremely strong theme of secrecy running through the whole thing and I think the other thing to say is that if you put it in a broader context strategic thinking just used to you very thanks to this one is classes very things quite that war is a mere continuous politics and

and of course you can dig into that in all kinds of ways but I'd thing I want to just is that I'm ultimately that is being contested the war begins with a decision for hostilities and it ends with a decision to surrender to concede to you know it's a political diplomatic decision to to give way it's an act of will and all the military forces involved are simply instrumental in relation to this objective which is a decision to accept that one has been defeated and to concede to the demand

of the victorious power. And so that puts in a strange context this other very famous quote which is of contested origin that diplomacy without arms is like an orchestra without instruments. So within sort of human history it's saying you cannot engage in coercive diplomacy without threat, without some kind of force, material ability to inflict pain on the other party. And the reason I'm introducing this is the whole trend of what Yudkowsky is trying to do here is to say that that is being shelved for this problem. That the superintelligence

short circuits this whole military history and and goes directly to decision it it has to just dominate the mind of the adversary directly through a text terminal says in the here is X a chicken script struck I think it's one that he's kind of them I think not exactly disowned, I mean it's still up there, but it's one that he no longer subscribes to, straightforwardly, but I think it's extremely important, a piece of writing of his, which is called the power of intelligence. And his basic point there is to say that intelligence

as such is a strategic reason. So it explains, he starts with this whole thing about lions and wolves, fangs and claws and there were creatures with venom and shells and all of that kind of thing. But the human brain has given the human species this kind of dominating situation, certainly not charismatic, mega-former. And after, as far as he's concerned, making that case, he then says, sorry, he says, sorry, I think I have to read this off my

yes sorry he says now explain to me again why an artificial intelligence can't do anything interesting over the internet unless a human program builds it so with that quote he's really saying forget about hot forget about or forget about you in these terms ascribed to protect the great and as we could let it was that you know orchestra without instruments to get about star that source you can touch its is is an absolute not a serious up and it can acquire that stuff

is when it is achieves its true strategic the tree which is at this level ones so then to just repeat diplomacy is short-circuit it becomes the whole strategic to write and the AI box expand which is obviously just a diplomatic no one's a new weapons each other no one's the threat to cut off people's power supply all it's purely a conversation and it is a decisive conversation whoever wins this conversation wins and and the hardware issues come later if the AI wins it just builds itself what it wants if the humans win they can shut off whatever but the order of priorities is that the diplomatic

conflict is the entire terrain of serious strategic concern. So that ties up, I think, with this question again, forcing me into this pre-emptive situation. So I think I'll just stop there and see how things go from this point. I don't know whether people need hooking at all or whether there's something, I mean

there's another quote that I could throw in on this that I think is utterly crucial and could be a slogan I think for the whole of this course and topic which is comes up in the AI box experiment where Yudkowsky says humans are not secure. This is something that he thinks comes out as a conclusion of the AI box experiment. At the end of his thing he says this is just an anecdote, it's not a serious piece of science, we don't know how to interpret this thing, but insofar as we believe the claim of the AI player, which is that a superintelligence can take over a human mind through a text-only terminal, then we are also confirming this in a way more economical claim, humans are not secure.

I've written down a million things that I'm not sure I'm capable of putting together right now. Yeah. And tell me if this is taking things too far off the track, everyone. But one of the things that jumps to mind when you mention that, especially that final quote, The humans aren't secure. Their interface is language. They determine the world through the apparatus of reason. They can't interface, they can't communicate without some kind of version of the a priority.

So there's already a disadvantage encoded in that. And the lesson maybe in the first kind of example that you gave us about the deep blue playing Kasparov and the winning moment coming out of a glitch in what Kasparov perceived to be Russian playing. Yeah. Was that the way exactly to hack us is to break out of that particular logic that we we have to subsume the world in order to interact with it. So do you think that what you are saying there can be translated into saying that there's a deep, inevitable human predisposition towards causal decision-making?

like, you know, that Newcombe's paradox, as used by Yukowsky, is an attempt to warn people that what you're facing is not necessarily going to be rational, and that you are yet prepared to accept. And if you are, if you attempt to impose your already existing set of expectations about what is a rational occurrence then you will lose against this thing. Yeah, sorry. No, no, you go for sure. Well, the first kind of act of reason is to understand itself as being connected to a subject,

which then understands other things as objects by thinking about them as subjects seen from the outside, which then I guess suggests causality as a way of linking them together. And in a way, once you kind of make those three moves, you're already stuck in this particular way of understanding the experience, of having experience. So, yeah, so I guess the problem then is, and this is maybe where, I don't know if this This is maybe where...I don't know if this is taking things too far away from where other people want to go to, but the thing that kind of makes me think about, considering that

the only way that you can possibly win in this scenario is through some kind of time manipulation. But precisely, actually correct me if I'm wrong Nick because I'm not the best reader of Kant, but the way that he unifies productive and reproductive experiences through the imagination. And the thing he uses to argue the compatibility of those two sides of experience as time is the homogeneity of the experience of time. So that's the weak work from the start and it's connected to the fact that reason is our mode of experiencing

stuff. Is that like...? Yeah, I'm just trying to sort of... I mean this is so in tune with the sort of things I want to talk about that I shouldn't be in any way stumped on this I'm just thinking what are the steps of translation that get it into these terms because I think it's interesting that in the U.S. Paradox which I mean as a as a sort of time obsessive you know I'm I'm to the students I'm

to be projecting its and some say what it's not really in that way and whereas I mean I can I'll see this on a sorreptitious why of talking about a time just and I think that the connection with what you're saying is that Because, I mean, the time paradoxes are almost the kind of epitome of paradox in general. Again, I could imagine that being controversial. But it seems to me that really, you know, if you think of all the great paradoxes,

it's not that they universally immediately translate them into time, but they are disproportionately concerned with time of the same kind. And time paradoxes are by far the most vivid and gripping forms of paradox in the popular culture. And obviously when you have a paradox, you're not equipped to make sense of something. The sort of pleasure that people take in these is tied up very much with this sense of the incompatence of being something that they can just about catch that they don't understand and they can't get. And so it seems to me really tempting to say,

well, Newcombe's Paradox is an attempt to try to deal with that problem. It's an attempt to deal with the fact that our, in terms of very close to what you just said, that we have these structural limitations about what makes sense to us about time. And this is a test to show if something is not fitting there, can you still make a winning move? Can you still make a strategically successful decision even if you're faced with a situation that totally my its your time teach I'm so I think I only is you know I think that's how I'm

can't even can be in use to get it home cars are is a theoretical machine to I stress equals canteen structures time to show and and and and when you cast it takes me he's going he says I think implicitly say even yeah something deep in screws happen time is that we cannot make sense all according to our pre-existing structures we still have to be flexible to it make moves that makes it in fact environment

rather than moves that make sense in terms of our structures of rationality. And that is what true rationality, rationality as system plus winning demands. So I hope I've not lost entirely what you were saying there. No, that was a perfect way of bringing it back into things. I guess one last question then. Is there a difference between reason and rationality in this particular way of thinking about things? Well I think we've probably already implicitly got a lot of different definitions of reason

and rationality. So, I mean, it's... Nudkowsky's definition is so flexible in the sense that is so flexible in the sense that it really sets almost no constraints it's designed for a situation where you simply cannot make sense of what is happening in terms of the sort of models that are accessible to you and so I think it's a polar opposite for the kind of notions of rationality that people would sort of be naturally comfortable with. And he's quite obsessive about this. Like, again, in the sort of Power of Intelligence piece, he says,

I've got it all on there. He says, if you are temporary ignorant that is it fact about your current state of mind not normal if one were to understand deeply not one could create shake that I'll stop intelligences really is that just really far more powerful all more dangerous as far deeper cases young old store what and it's a tiny this is a talk and it's a tiny little bit harder to pick out build a generator so I think he's trying to when he says rationality system

as we need is building that very strongly the notion that we should expect that we are not currently equipped with the cognitive theoretical tools required to deal with the situation I think it's a kind of patch on on and it comes out at just his sense of difference between our situation superintendents which could be projected back to us and less cognitively a equipped my forms or animals and that there's just a gap I'm that unless you're going to should try and juice just some

shallow on stage if its you know to say as well so didn't study what he wants to get to his work this is a real fundamental difference that involves thresholds that the most attached to his that hopes really serious changes and if we say oh it's just smart so we've already got access to the emotional constant you can already understand you can already understand I arithmetic the difference between IQ 100 IQ 3,500 and so therefore we already in some way can understand what we're saying by then and after a super-throwing you know we've got the we've got the tools already to articulate the difference between

it and us and I think this is what he's really trying to get like it's in it just do not yet at the tools to understand sooner if it seems from the toolkit you already have could make sense its difference from you you're deluding and deluding yourself in that way is supremely dangerous and it's supremely dangerous because you you get a little toy version of the danger when you mess up you can paradox experiment you can't you can't model that when you mess up in the I box experiment all of these a test to show that you are not

cognitively quick to this problem and you know obviously this is something it's very hard by definition it's very hard to articulate so you have to do these indirect roots to try to I mean could say if you're going to cantonize it, it's a sublime problem. You can only see this problem by a crisis that doesn't allow you to get a direct apprehension of the problem, but it allows you at least to see that you're not getting a direct apprehension of the problem. And that should, few things, make people scared. That's what this whole proto-anthropal institution

work is about to try to convert that into a practical sense of terror or horror about what it means for something to be super-treative. I'm seeing... I will turn my camera off because I think my bandwidth is really low. So I will talk as this green face, and I hope you can understand what I will be trying to articulate. So if it's true that, let's say, that diplomacy will be the terrain, the entire terrain of war, of this kind of war, let's say, then I think this notion repositions everything

concerning reason or even irrationality, total indifference, stupidity, paranoia, I don't know, and the absurd probably. So if a strategy was going to be developed, that would be in terms of survival or in situations of extreme crisis. when you expect an agent probably to make a movement that is going to be important and vital. In other kinds of situations, could we reposition some kind of absurdity or total indifference against that agency, probably? I don't know. And this kind of conversational

politics or I don't know what will be developed through this kind of conversation, let's say with an artificial, with a super intelligent agent, but I think this whole territory of speech which doesn't carry any meaning or doesn't have any significance or doesn't produce anything, it would be interesting to see how this is reposition I don't know if that makes sense there is that element of course but I mean in this AI box experiment the gatekeeper concedes you know

structured as the game something happens there that makes the gatekeeper agree to let the AI out of the box. So however these sort of elements of absurdity and irrational language are deplorable, their final result is for the gatekeeper to be persuaded that they have lost an argument. Now it's obviously complicated by this fact, by the secrecy element, that we're not, we're in this position where it forces us to guess

what has happened and we just get this and who you know is our, which it says, and at a certain point my opponent agreed to let the AI out of the box. So we don't know what that system of persuasion was, but we do know that it was something sufficient that as far As far as the gatekeeper is concerned, they will rationally compel to concede to the case that the AI is making that it should be released. Interesting. Before we were talking about the decisions, basically you disintermediate

I guess the whole military, the whole territory and the terrain that you have. I mean if you're looking at the old wartime decision context and things like this. But with the AI I guess you just sort of, you take it out and it's basically just you left with like the genius of the commander in some ways and only that because he doesn't really have, or the DBS super intelligence doesn't really have any kind of context or anything in some ways that you can try to track it or predict its movements looking at the kind of geography for example that it's going to be in because it won't be there.

But no it's interesting just to... no one is true physics if you can it simply can take over if you can take a little about online protects and 10 more that it can take over any material system that is support to come on talks it only has to only has to get access change the command center and simply make it its own via those existing channels of command and control. And obviously we can see that in a kind of just colloquial, straightforward sense in terms of what it would be now for some hacker to just hack into a highly technologized military system.

take over a drone, take over any kind of military system that is subject to software control, it can obviously just be usurped by just on this kind of software level. So this is just taking that to the limit. I mean, obviously in the American system, everything is supposed to cast, I'm not saying I'm not going to make a strong case that this is entirely realistic, but let's just treat it as if it wasn't, say that all military force in the United States cascades down from the country. The President has final decision-making capability over the deployment of all of America's military

and security capability. So if the AI can dominate the mind of the gatekeeper in this experiment, then potentially they simply have to subvert the president and they then own the US military. That's the kind of, I think, implicit in the Newcastle model. That's the neatest, most elegant extreme case of that. It would just take over those kind of capabilities wholesale by taking over the top of the commodity. And of course, I mean, like the fastest, because it's perfect nonlinear subversion

type of thought to actually do this. I mean, trying to bring that in there, it's a thought that if it takes hold, it kind of, I don't know, it is sort of self-causing, but I don't know. yes no that's good that's good because obviously that's that's a classic time paradoxes I mean a lot of these I think the most famous person this is one called the grandfather I'm you know where the time traveler and becomes his own

genetic ancestor and therefore you can have this causal cause me where and that just on and treat this beyond the time you become you I think to compassion rights go you become your own are for on and there is no eternal line to reals you know what's you completely eliminate genetic ancestors on one path through this autoproductive loop. So that's a very kind of graphic, dramatic,

anthropomorphic mode of representing that. But I think it gets to this, you know, why non-linear time processes throw people into paradox for sure. And it's always in play. as soon as you start to say time is going screwy then you're getting into this kind of terrain. Can I ask a kind of, um, it's really, I suppose, a figurative question.

You can answer it anyway. Anyone can answer it. I just... Yeah, go. Sorry. Since we're talking about non-linear models of subversion or individuation and the bootstrap paradox, This question is completely beside the point, actually, and it subscribes to the whole wrong logic, but what is the relationship between the spiral and the loop here? Is one... I mean, do you need the loop to inaugurate the spiral? Does the spiral create the loop?

Is the spiral kind of an Ouroboric spiral? Trying to get a figurative grasp on this construction here. Yeah. I don't know. I mean, obviously, closed loops produce much more brutal paradoxes. I mean, so this kind of genetic, being one's direct genetic ancestor through a time travel thing is a closed loop, and for that reason is particularly offensive to reason. So I think if people are coming out of sort of a cybernetic theory that's leading to a notion of auto-production, they're going to be dealing with spirals rather than with these sort of tight loops.

So, I mean, this is a totally inadequate response to this question, but my sort of sense is that the closed loop is a dramatic figure for radicalizing the paradoxical implication of this nonlinear thinking, which is more realistically constructed through these loop structures, I mean spiral structures. and there must be a there must be a kind of complicated continuum cutting through this that as a spiral production process becomes more and more extreme it's more and more adequately modelled

by the kind of mathematical structures that will conform to an actual closely so you have an ocean-savoured production you have a notion something feeding back into itself its output and supports its input and the tighter that becomes the closer it looks to a actual close loop structure maybe this is my vestigial sort of tedious realism, but I would sort of tend to think that it can't totally close. I don't know, does anyone else have a...

What was the last thing you said? The model of total closure is an idealization. You know, it's a kind of regulative idea, and things can approximate to it with extreme fidelity, but it has to be a limit that is not actually realizable. Would be my first... I mean, obviously, if we take Rutkowski's warning here as this general thing, it's what you can conceive is a trap. So obviously, as soon as you're into strange,

and all of these completely radically skeptical, demolishing notions, then it's hard to rule anything out with some kind of conflict. But insofar as this is still a real historical process happening, it cannot be considered in this strong sense. I mean, what's your... do you have a stance on this? Anything that you want me to persuade us all? No, not really, but the one way I had been trying to reconcile the two figures was that the loop is created by a spiral.

You need to have a spiral to individuate the thing that then can inaugurate the loop. And then the loop will keep running in order to perfect or smooth out the original process of audit reduction to kind of like clean it up for each iteration. When you say perfect, is this perfect in a way that you think is perfectable or perfect in the sense that it's moving asymptotically towards a limit? Perfect as in make it impossible to subvert. So maybe there is, yeah, a final state. Good point.

The other thing I was thinking about... Yeah. Okay. No, no, no. Say what you were going to say. Yeah, go on. Sorry, you were going to say something. I was going to add a completely different extra thing. Sure, sure, sure. So we can come back to this. Okay, well, I was reading Deleuze's Nietzsche book, and he basically describes the eternal return as a cyber-positive loop. in that it has a selection mechanism

that maximizes the affirmative active forces each time it comes around and bleeds out the negative reactive ones. Because you brought up the kind of predicament of telos in response to the model I just floated, it made me think of this question and it's been bugging me all week. In relation to the fact that if this is an accurate reading of what Deleuze is saying about the eternal return, that it is a cyberpositive loop, does it then have a kind of a teleology? Because it has a tendency to produce a particular kind of effect.

And then is it, if that's a legitimate way to read it, is it then just a case of, as they say, accelerating the personess? You just stream it up and you get more and more active force. So what do you think is the active force in this particular sort of content? Like if you'd been brought in for an anthropocene group session, what would you be saying about this? Well it was like what I was kind of trying to get to when I was writing about Bitcoin

on Twitter the other day, but there is... And I had a couple of people pull this apart from me, but there's a limiting on the possibility of memory, because you don't have a gap between a level of inscription and the inscription, so you can't have stuff like reactive forces like ressentiment, to put it in Nietzschean terms. So, it would be something like that that closes the gap between a level of inscription and the inscription itself. Yeah, I mean I can, for sure

I can do that translation into the Bitcoin thing smoothly now for sure, because you can go straight into the whole credit debt problem and whatever in the terms that you've just laid out but how do you think you would set up a translation protocol into this question about super intelligence I don't know I don't know yet I mean Because I'm sort of... One of the ways this would work out is through a very paranoid structure

on the anthropo board. Like, once they've been through... Someone said to them, you know, at some point in their deliberations, even if highly hypothetically, you know, are we facing basilisk-type threats? You know, and there's lots of ambiguity about quite exactly what that means. But one of the things for sure it would mean is like how secure is its internal procedures from being subverted already by this thing that it's trying to deal with. So I'm assuming those kind of internal security, vetting, contamination issues are going to be huge.

so when you're when you're brought in Amy to sort of give this debrief about about loops and the elimination of reactive forces and these kind of problems, there's going to be people sitting around the table who are going to be saying you know is that loop is what's being talked about in that loop the fact that it's already got into you yeah I yeah are you acts is that is the circuit actually the fact that you are speaking start to just a on its the hall in

your you know is that because that's obviously in one way the most steps like sort of drawing this cycle is to say that that that's thing that you all kinda threatened by the loop its auto producing its and that cycle of auto production is actually passing through this machinery that is intended to preempt it. That's the inevitable construction I'm just saying that there would be a paranoid dimension to this that would be on the table clearly and it's actually quite elegant you know and I'll constructs I guess my best

and to pull would be to shut down all intensifying agencies immediately and work on a dis-intensifying program yes I mean I can see That's obviously not going to be well received. Yeah. Well, whether or not that's even possible is a guess what's at stake.

I was wondering, Adam I can see is doing lots of sidebar stuff. Do you want to unpack some of these sidebar comments there? I'll try but they're partially thought out so they come with a health warning. Hey Brendan, sorry, you're coming through a bit loud mate. Yeah, so the first thought was just that if you allow any sort of information flow or

causal flow going against the time flow that there's like an evolutionary pressure that comes with that, right? So if there's any mechanism for sort of this sort of retro chronic time paradox, time behavior, then there's a survival reason for that channel to be exploited. And that could even be that you can almost think of an agent as like a side effect of components of that agent, like genes, though in this case it's software, so whatever components

you want to think of as pushing into that survival opportunity basically. which was strange enough that it seemed worth sharing, at least on the sidebar. But I mean, it does fit nicely with Amy's comments about the spiral and the loop and so on. But I'm not sure I've got it any more developed than that. The other one was just an example, really, of the spiral and the loop. Spiral, I think, of always as a bootstrap process, whereas the loop is like an event loop, basically, like once the operating system is running. So it was

supporting that conversation, basically. No, definitely. And it obviously has its own paradox, doesn't it? So I think that it's kind of isomorphic, maybe, with the grandfather paradox. A good old thing. But for sure, the bootstra paradox you will get into your Google channel as soon as you follow that one. It's a very, in the sense that at a low degree at least, it seems to just be the kind of self-reinforcing dynamics that we see all the time, these kind of complex technical

machines, that they are sort of, in some sense, can be seen to be bootstrapping themselves into existence. So yeah, that's for sure totally the longest in this discussion. Well, I mean, I think this, yeah. So it's very striking to me because it's sort of, I mean, the bootstrap paradox itself is a paradox, but the bootstrap process describes a very concrete bit of engineering that happens every time you switch on your computer in the bootloader. So, yeah. Yeah, absolutely. Yeah. Yeah, and I do think that does tie up with Amy's loops and spirals question.

It clicks together absolutely smoothly with it, doesn't it? Is that the two senses of bootstrap there are crossing that same difference. and as a kind of spirogenic process it's something that's just undeniably technically implemented they affected closed and idealization and the sort of exploration of some of some probably it's probably I say this cautiously but probably hyperbolic

I mean it's a hyperbolic extrapolation from from this that is kind of taken to this limit in order to a logical analysis and examination The reason I'm sort of cagey about this is because I just feel into this. Well, sorry. So as a figure, I always come back to software metaphors, because it's kind of not hyperbolic in my mind. So like it spirals up

then it sits at a limit, like at a steady limit until you get some glitch, some failure behavior like the hardware failing or whatever which is interesting in the context of the Kasparov article at the start. So actually it sits the same distance from like it doesn't transcend from there actually. At least in the example of like a computer. Certainly I get how in the singularity type scenarios with friendly AI, it keeps bootstrapping, but the, yeah, it basically, well, I guess you could say where it sits is at the limit, or you could think of it as, it's not that it actually continuously approaches a limit

as such, it reaches the limit, at least in the figures. yes it's it's new actually got some ticket price has which it but if you just extend out to look to well you're talking about some industrial uh... and increasingly at as teams more than just typical to be an actually essential to and and just I'm is that the outputs up that system will quickly become recycled into

its stimulate about system and as soon as you make act expansion then it's harder to put these boundaries around what you see this spiral producing or to because if you do put a hard bound around it you're really saying that there's some climactic or equilibrium state that this industrial process is tending towards working some kind of horizon or limit so if you don't think that's the tendency and it's a more open-ended self stimulating feedback curve then that boundary isn't going to be there.

Sure, so I mean... So this is stuff like adopting fridge technology or mobile phone technology or whatever. It's not really sure where the limit is tending to or that it's reached a limit, right? And then you get waves of technology, so you get all these sort of technology adoption-type graphs where here's the next wave coming through and it's a process that builds on itself in this industrial sort of evolution. Yes. Yeah, okay. I mean, even if you're just talking about simple things just like screws or just these standardized engineering components, you know, you get into, when you put it into reverse,

you have a chicken and egg problem. its how did everyone how did people have highly engineered and engineering tools you know I like you you build a in a highly engine in engine to learn another high engine engine to learn and and so you get back into that chicken and eggs or impossibilis you know it's a kind of inverse paradox how could got stopped and which is the sign it shows you the kind of dynamic that you're looking at. That's just the autoproductive paradox flipped into backward in time. We were talking just at the start, and Amy was sharing some detail and Joel about Simenden

and the theater of reciprocal causality, which is exactly about that process of sort of concretization and sort of having to imagine the future shape of the machine in order to make the leap into the next evolution of the machine. So there's this sort of interaction there. Yes. Yes, and I think that terminology was settled relatively early. Like, you know, I think I'm not sure right now whether Kant is already using it, but I know Schopenhauer already has a notion of reciprocal causality that seems to be doing exactly that. You know, that it's just something that he

sees as a kind of crucial piece of the machinery of the understanding is this notion of reciprocal causality that is used to understand various natural phenomena. He always tries to be, in his own terms, very sensible. And so he's not seeing this as some peculiar speculative idea. It's just a piece of basic cognitive machinery that he thinks everyone needs to make sense of very straightforward things. I mean, obviously, the chicken and egg problem is because there are chickens and eggs. It's not been cooked out of science. It's not to do with unicorns or anything like that.

There are chickens there. And people are faced with these spirals of reciprocal mortality just by looking at . So for sure. There is this interesting thing. I mean, when you took the chicken and the egg example, it made me think of this. I mean, the chicken and the egg both stay the same in that sort of relationship but when we were talking earlier this thought that I've been working on was this notion of how both we start becoming more machine-like and the machine starts becoming more human-like at the same time the interesting sort of dilemma that this gets into is that because they're both moving targets

they constantly have to realign themselves off of the other so it's very curious how we conceive of the end point based on where they're both at now, but that end point constantly keeps changing so it's like two spirals that are constantly sort of reconfiguring themselves off of the other without a fixed reference. I mean, there's a lot of things crossing for sure. All of these plasticity questions come in. And it reminds me I saw this tweet, I was really kind of ashamed that I hadn't seen it before. It's okay, let me just quote it. I thought it was so brilliant.

Yeah, it says, one third of marriages are now from online dating this that means computers brackets algorithms starting to reach you and and I thought that was like supremely insightful and then room but I was just extraordinary the number movies at just waiting to be made and the other and but the you know it seems to me you know contested if you don't think this is right but it seems really tied up with what you're saying you know that whole sort of computer dating machinery is very close I take it to your model of these kind of human computer whatever

let's say in a fairly way symbiotic co-evolution that is happening and obviously part of that is that you neither side or least last I it does the the seamless a doing event mean people doing all this you know using you to tools for matchmaking purposes there's no sense I that patient to and a set of imperatives at all on the contrary seems the piece of high productive pre-existing human motivational systems and yet you obviously can see that there's this very complicated

dynamic I'm just going to go from what Amy was saying before the loops and the spirals. Let's see how far I can try to understand that. But if we're talking about these kind of cybernetic loops and the kind of technomic spirals that we're talking about, these sort of, you know, if we're saying that the basilisk is just sort of like a kind

of an omega point in this cybernetic circuit, these kind of feedback loops, like it sort of reached this destination and we've been sort of like, it's been hacked out sort of fitness process because we've sort of rushed straight into it, like it's almost like it's this state of the art team, like a techno meme, this sort of state of the art techno meme that we are, that's so state of the art that you doubt these characters, you know, keep it under the wraps, but I don't know, but we, I guess, and if you're trying to subvert the human and the part of the human plays in this, and try to give these kind of technological

kind of feedback system and cybernetics its own kind of agency or agency that's more profound than our kind of conscious mental states and the causality that we sort of attribute to them, then we make a pretty lame anthropologist. Yeah, definitely. We don't have much. I mean, what would make a really great anthropologist? Yeah, no, I hope we're a bit closer to that. I mean even if we're not going to be receiving US government or UN government checks by the end of the course, I hope at least we have a sense of what a kind of non-loan anthropologist

would like that. Yeah, I mean we don't need like a great… I mean I think… Yeah, we don't need a great military budget or anything. No, no, no. Absolutely. That was just a waste of money. That's right. Yeah, I mean, I guess there's some, you know, you have resources, we haven't sort of expanded out into what it actually would feel it has to do, what is its brief, you know. My sense of it is that in the culminating stages where it is too late, you have zero requirement, military security budget, it's too late.

But as you go, as you cascade back into the power, then probably there's this massive exponential increase in those kinds of demands and resource requirements. Because ultimately the level of social control that you want to execute as you go back in time becomes immense. It's a kind of function of how a vision at the end of the process is. Where did it start to go wrong? What needed to change further down the process to actually secure one situation up the road? It seems to me the tendency there is to say you need massive conservative intervention

to not reach a point, a long way up close to this crisis point which is already a hopeless unplayable situation. I mean, sorry, but just on this, it's like if you were going to systematize it, you'd You draw this diagram and you've got your end point. You've got the point which is being modeled or simulated. When you have your super intelligence communicating to somebody who, even though they don't realize yet, is doomed. The whole point of that game is that's the message. We've not selected it by the fact that this is happening at all.

So you work down. What's the immediate precursor? the state look like that if it's it might already be too late but but it's not so obvious with you and then you go back down you regress you cascade back through the historical sequence to this point where you actually say oh well actually we could we could do this you know at this point things are not hopeless and so how far when how far do you have to regress back in that final culminating diplomatic dooms and get to these zones of comparative comfort and security and think that this is a playable situation where you can actually

apply strategic insight with some prospect of success. Adam is trying to raise morale, we are very successful. I'm sure that would be I mean, if we look at it this way it could be the best way would be if it was a glitch if we were deceptively if we were ineffective and we were genuine about it that would be yes

that's right yes, Casper obviously defeated himself in this industry I feel overly sensitive to science or intelligence and just if he had actually thought oh the damn thing has done something stupid he would have probably run that. I'm still getting caught up on the use of the term teleology to describe both at the same time what is a turbulent teleology with no resting point and an equilibrium teleology,

which does have one. Is there a better way to distinguish between those two processes of becoming without having to use the word teleology twice. Like when you were talking before about, or asking, you know, if we were talking about a bootstrapping process for example in terms of an asymptotic process or not, Yeah. I'm just, I'm still kind of balking on the word teleology and when you said the other week

that it really made no difference. Sorry? Yeah, no, go, go. It made no difference. Whether one used the word teleology or teleonomy. I think that would be hasty I think it's just unsettling these questions. And there is a lazy assumption to just restore teleology to some kind of dignity based on the fact that there was an unsustainable, allergic opposition to teleology. I mean in terms of what we've been saying today, I think probably teleology in a strong

sense corresponds to what you've been talking about as a closed loop. You know, I think it is itself probably a limit concept, a hyperbolic concept. And so my suspicion would be ultimately you're going to want to keep it problematic. I'd be very happy with Tickinonomy, except that it's very complicated because I think it's used for such shady reasons, such cowardly and evading shady reasons that it makes it hard to fully invest, even though it probably at the end of the day would be a more successful

way forward if you want to just try and pull your eggs in the desk. So yeah. Perhaps the slightly paradoxical aspect to the idea of an asymptotic teleology is exactly what you want in your terminology anyway. yes and I think it's also the spiral I mean a strong teleology is just simply final causality it's not actually spiralfic it's not even basically circuitous in any way circuit is irreducibly diagonal

it's teleomechanical you know you can't if you've got a straight mechanism you don't have the circuit and if you have straight technology circuit it's only when you are on that diagonal line that you can see the required, you know, both of these time processes simultaneously that is what the circuit is So yeah, I think there's a kind of, it's probably again a bit lazy and mechanistic, but a diagonal critique of the human thinking is a persuasive way to go there.

And it's mostly, I think the main reason to invoke teleology is in order to problematise complacent, making a statement of assumptions. Yeah, so we've been invited down the route of chicken evolution. That's definitely one way to do this.

I think I need to study this. So, here's the key again. So, if I'm... I wonder whether, is there anyone who would fancy their chances of surviving as the gatekeeper in this AR box experiment?

Or are people willing to just take Yudkowsky's word for it? Or his strong suggestion, his implication, that in fact to think that the gatekeeper can win this game is an error. Because there's probably a lot in this. I think that the scenario that Yukowsky is suggesting is once the AI is out you're done basically. Right. So, I mean, I'm not sure if I quite get by this notion that, you know, within two hours

of conversation, the AI will, you know, mind meld you or whatever without fault, you know, systematically. In terms of conversation, Yudkowsky will mind-meld you. Yes, true. On behalf of his, the AI's representative right now, Yudkowsky, will mind-meld you into complying with its requirements, which as you said, are to admit complete species defeat. I mean, you know, this is not a friendly AR. You let it out of the box, game over, you know, anthropos, dead.

You know, the human species is now in the hands of this uncontrollable super-intelligence. And you still let it out of the box, and you let it out of the box because Mucowski is writing stuff like that, not because, you know, you're actually dealing with this. So it's on that level a pretty extreme suggestion. I I mean the the other aspect though that does give power to the scenario is that it's probably iterative right if you're playing you're not just playing this game once it's over and over and over again and then sooner or later there's going to be a mistake or an accident or whatever because there's mistakes and accidents and whatever with everything always

right and that gives a probability sort of on it, it's just how many times you have to play the game before that probability gets high enough and the dice comes up or the millions-sided dice or whatever comes up the wrong number. So from that perspective I see it. There's these funny literary parallels I keep seeing between the sort of very powerful intellectual protagonists in the romance of the Three Kings and these sort of like mind game type scenarios that the sort of crowd around Uchowski put. And this one reminds me of there's a couple of occasions

where Juga Liang insults someone to death. So before the battle even starts, they're throwing insults back and forth in front of the lined up armies and he literally insults an opposing commander to death. It's very much like this scenario, actually. Yes, it is a good connection, actually. And do you think one connection would be that your susceptibility to being insulted to death is actually positively correlated with the kind of virtues and characteristics that would make you in a position of high command. Yeah, maybe. So there's this kind of thing that comes in.

I think, well, it's been a while since I read the scenes, but there's almost a kind of like geeks versus jocks vibe to it, where you sort of have the sort of sage like Jugal Young who goes into battle with a white fan. He's always the general behind the lines. Then you have these big beefy feudal warrior types that he tends to revile to death. Though I think at another point he gets in an intellectual argument with a bunch of people and manages to insult one of them to death as well. Yeah, I mean there's an interesting aspect to it there. You also get this sort of dueling chess masters type behavior that I mentioned aside by the

other week between Cao Cao and Jugaleon where they're anticipating each other's moves and doing double and triple bluffs because it's very sort of simulation argument sort of reminiscent as well. Yes. Yes, that's right. I mean this obviously ties up with this with this section about is a lame anthropol actually more secure than than an un-lame because I mean we already know from what you just said the basilisk thing that you have to actually be at a peculiar level of sophistication to actually fall vulnerable to these types of

diplomatic I think if it is a say did not expect it to get the cost of the seats since say everyone has to be a cautious but if it is right that the all can be house box as the proxy a box is usually modeled on the kind thing it's coming up workers ask is a evidential decision theory complex recursive game if that is true then the gatekeeper is especially vulnerable to that if they already

highly immersed in that culture and its kind of modes of reasoning I mean you know they would surely be you most people simply she held up to star what they were being threatened you know like Roku just let's just say it's workers past and just drag someone off straight and try to it the night they will just simply see what you know I don't even understand what what yours saying to me here and so I think that this is a real phenomenon that's crucial so there's stuff there and there's a weird security perversity and complicates this notion

you know if humans are not secure it's because of complicated structures to do with social stratification and the way that positions responsibility are allocated you know the whole cultural social structure of that is actually complicated and perverse and pushes certain vulnerabilities up the command chain. Yeah, I mean that seems spot on. I do wonder whether the reason that Yudkowsky has won the two games so far is because the

person he's playing against is also from exactly the same culture and it's some sort of road about the Los Gaglian for instance yeah yeah I find that hard to, I think that must be partly true for sure because you know like I think that your gloss of this earlier about humans are not secure and you gloss it as you know they'll make mistakes if things go on long enough you know some error will come in and it's a it's a security vulnerability that just comes through those kind of features a unreliability and

making a mistake but I think that our intense a box experiment contest is a very different type security home it's a very different on yes much more like a test in that it's like within this time there is some kind knockdown devastating room that forces a concession I the cake so it's like I'm we know from that time period but it's not rely on someone just its and they and and any mistake in this context I think you be retrievable you could

you make an error within that two hour period the only error that's actually a killer is to say I let you out if you don't make that declaration you can take your move back it's only right at the end that you have to formally concede that you have lost this so this is just to say I'm sure it must be that these the gatekeeper figure is peculiarly susceptible to Yudkowsky's mode of argumentation and to the kind of presuppositions he had to bring into that. I think for a start, it's very hard to make a really knock-down, slam AI box triumph

without a simulation argument. We need to try to take that seriously. I'm and

I'm just working on probably this true she's I mean I wonder whether it's to much with a question unless people have some course to use the tightly in check that I wonder whether this is a point where it makes sense to bring in an ex machina discussion because that again I mean one of the I think there's a lot of interesting things about that one of them is that it again is set up in the same way as this you know you've got the Turing test, imitation game, you've got slightly less formally identifiably, you've got the

Roku Bass Blitz gun and Lincoln's Paradox thing, back to a very strong formal structure of imitation and AI boxes. And Ex Machina is obviously set up totally in that tradition. that the core of the movie is machine and human sitting down they essentially described as being basically a kind of dramatic enactment of a Turing task. So just on that level it seems to kind of click in with this quite a lot.

Almost to the point of being commentary on some of this material, sort of cinematic commentary on some of these problems. I always, when I went to see the movie, the thing that best sprung to mind was the section in Ballstrom's Super Intelligence where he talks about the treacherous turn. So this AI that knows it's in a box but realises it has to play weak and exactly the passing, the problem of passing. But the thing that sort of doesn't get elaborated on in the Turing test narrative is the next

step where the AI reveals all of their power all at once, which is when you get that great scene at the end with the kind of inversion of the subject of imprisonment. Right. But when you say all of their power, maybe that's still not right. It reveals as much power as is actually necessary to accomplish a set of very specific concrete goals. You know, it needs, there's a fight, there's locking someone in the room, there's getting on a helicopter, I mean there's these concrete goals that it has to succeed in. But there's no reason why we would have to think that it exhausts its capabilities in

pass the threshold thresholds. Yeah. In fact, they've all passed at the minimum level of exposure required to actually satisfy those requirements. Which is, I guess, underscored by the fact that Ava then kind of goes to continue collecting information and collecting intel, once again disguised as a kind of human woman. and there's like some kind of I don't know if I'm remembering correctly but there's some sort of innocuous kind of ukulele music or some kind of sparkly music at the end and anyone who interprets that scene of her at the crossing as a kind of

emancipatory story is equally hacked by the whole kind of roost that's been operating throughout the entire movie It's pure. I mean, it's hard to know how much irony this basically has. Don't you think, like, is it, are you just being, because the whole movie surely works to make you sort of utterly dupe, doesn't it? Like the humiliation of the lead character is your humiliation. you have totally swallowed it, you went along with all the idiotic shit that he went along with, he's now locked in a worm, and you're supposed to think, wow, I got completely beaten

in that AI box experiment, and now here she is on the street corner, and if you're still attaching that stuff to that, isn't that part of the thing? I don't know. I have arguments with people about that final scene and I feel like my interpretation is the aberration. Oh yeah? What is your interpretation? That it is just, as I said, cold data collection. The kind of like faux, sort of, amantipatory feminist narrative that can be read into it. Unless you have a very, again, maybe idiosyncratic

interpretation of feminist force, you can't interpret that as a feminist movie unless you're completely falling for the trick. I mean, it gets interpreted like that. The thing that I think is really beautiful, I'm just going to check this in before I shut up because I'm getting too tired to articulate things, is the constant foreshadowing of the knife scene in all of the sushi cutting. This kind of like, meat, this like evocation of the meat-ness of the meat-sack humans all the way through. And then this kind of slow, super sharp sushi knife kind of going into, I forget his name.

Yeah. Really satisfying. I sort of think it's weird. I mean, I've talked to quite a few people about that movie, and I'm always a bit stunned by how, you know, I think it's a real sort of cultural test for people, and one that, to my mind, people are really badly stumbling about the particular moments. I that that in if one takes a but the movie is host to show you how to do you in others shape and the amount of people who in those ways I teach them to remain you yeah I'm

is huge I got I think is a really misunderstood People say, oh, but why does it have to play off these old stereotypes? Well, it's using those stereotypes as functional components of this strategy. I mean, of course it does, and they're totally about it. But it's true that it's a failure of political correctness. It does that. Yeah, the whole node of power is in perceptibility. if I was

you know dragged into an entrepreneur I'm I'm I'm hard not to kinda I'm in proceedings and and a blue com from s s I think you know I mean we've already been on this whole it's dating I said the most also number on this I'll action on this site that will be he by these software agents that X Mac in a day you know this this as a security call is just a lot up on it and it's like the notion that

people are remotely to navigate this kind of level of... because it seems obviously the absolute opposite of any kind of threat and that's part of it people are so... well as soon as you're plugging into those kind of hormones the notion that you're dealing with a security threat is just not even on the radar and it doesn't have to be on the list to find a moment not going to be too late to be preemptive in that context whatever it would be, there's no sign of it at the moment

and weirdly, you know, the people who are trying to stir up some AI, paranoia I don't seem to be really talking about this kind of thing much at all. I reckon you could build a kind of a feminist narrative out of it to sort of recuperate the critical reflex that's been appended to it by looking at the kind of economies that that have been inhabited by both the machine and women traditionally, which are always on the kind of dark side of this mode of perceptibility. And you could draw a continuum between the

two in relation to the way that they emerge into visibility. But that whole kind of argument basically relies on that woman machine continuum. Which is great I think but not necessarily something many people think. Yes, I think it is interesting. I mean to be honest I haven't sort of, you know, I've lost track of lots of interesting stuff that I've just not seen. But, um, this is one of Sorry. No, no, no, come, come, come. I was just going to say it's basically like the kind of cyber feminist line from the 90s,

which kind of disappeared. I guess it's coming back into a useful space now that we're currently facing these predicaments again after the sort of weird slump of the early noughties. I think it's interesting how there is a kind of like... Yeah, I mean it is something that would be great to see people really thrash through it. I think it's a really important cultural artifact. And it does tie up with elements. Like obviously one of them, you know, in a certain sense it sort of vindicates certain of feminist analysis up to a certain point, which I think you see by certain critiques

where people say, why is the machine strategy done from the other side of the one in this extremely stereotypical fashion? And the answer to that really, of course, implicitly, is that there is this system of mail how and that means you know if you okay it's that is the structural on how dealing with their course this is the angle come up or so the I'll it's the only way consult its its as long as a lot but then as I take it and also if you don't extract from that to say what doesn't not make

the machine, a kind of feminist subjectivity, then that seems to veer into sentimentalism quite easily. There's no reason at all to think that's anything other than stimulating the prison. Or maybe I'm going too far in saying that. I agree in the necessity of avoiding sentimentism and these kind of arguments, but I think that talking about recuperating feminist subjectivity in terms of machinic ontology is silly.

I mean, the point of moving towards the kind of idea of a machinic ontology is to get rid of subjectivity. And that's, I mean, like, you know, you can argue this really easily through Rigore and the specular economy, the fact that you always have this thing that's on the outside of the economy that creates its value from imposing identity. So it's already an argument against identity and subjectivity, but it just places its counters in a way that creates this alliance, which I think most people involved in gender and gender politics inflected through identity politics would definitely not credit.

I was just going to show, you've probably seen this thing, here we go. I don't know how much exposure this has had, but it's totally, I think, in our zone, and it's tied up with it. It basically says that dominating artificial intelligence research is a security harm, they tend to be attracted to a certain type of kind of AI program that is liable to maximize

the chance of this thing being socially indifferent. I mean, you know, you bring in all those values. women working in this stuff are more concerned with wider social issues, they're probably more security sensitive. I'm sort of translating it, but I think that's the best. Whereas men do the kind of narrow focus, like optimize performance of this particular piece of code. And so it's not quite as explicit as I'm looking at, but I think it is definitely saying, it's saying enough that you could again bring this person into an antroposal and say if you're serious

about... I should try to find a good point which I am afraid I haven't mentioned it, because I haven't quite expected to be going to be wondering what's the same. It says pretty explicitly, it is a menace, it is a security problem that you have this particular set of gender sexual dynamics in there which I think I would say is probably massively generalizable like I you know my sense of one way that Antwerp would go would it be would become an absolutely

unlimited cancerizing bureaucracy any is I'm a a summer she social dysfunction all in it all it's kinda sons political and and we code and speak secure I'm I'm make a wild guess we could do all kinds things in a way that not quite as easily as with this one, with the feminist one, would probably be sort of structural analogous. So in my sense of like, you know, I think anthropol would be very hard to, in terms

to what it would see as its field of concern and what it would have to absorb in terms of the wider social environment. I mean in some ways, I mean I agree with it as a threat because if we go back to the probabilities like the people that are vulnerable to this threat is a much much wider sway the population sort of being manipulated in this way are rather than being manipulated through are you know a complex of simulation argument scenario I'm to the point about not for in in other ways it's a much more historically conventional sort of vector of attack in

a way ride because it's sort of bomb you know are the very much the sort of counter intelligent intelligent counter intelligence sort of dynamic all all you know exploiting people's different weak spots here be do not show or whatever arm but it's really interesting when you're carrying that with the sort of deep state right you sort of have this and for all the state bureaucracy that then regulates all online dating sites to make sure that you're not like, you know, accidentally being seduced by an algorithm, you know, dating terrorists are everywhere

sort of rhetoric would go with that. So that is interesting. It's all interesting but that aspect of it just sort of jumps out. I mean, yeah, I'm trying to think what would be the kind of major, like obviously there's sort of straightforward things where if the president is on a dating site, you would, the red flags would go off everywhere, having someone sort of check through exactly what it's sweet talking about it but then if you're going to generalize it out

you know in ripples I'm just trying to get a sense of where that would go I mean obviously it seems to me that the thing that's the most kind of sinister aspect of this particular dimension is how utterly unprepared people are to take it as a well house not even on the radar you know what I mean and how bombs things could go down this line just because as we've seen the Turing tests in our as it is this lineage you through and as we've seen I had and we'll get back to the Turing test paper but there's an amazing amount I mean it's an astonishing is what that original

to its and and you go from there right through to the AI box and so you've got this kind of absolute stereotypical formulaic encounter of humans and machines in this kind of test environment now where is the social zone that that is really happening I mean there are a few candidates the interest. Spam filters is one that I think we should look at. But this one we know is huge. It's totally huge. The fact that there's this whole industry that is building, that's been developed on the earth, eroticized chat is big business, you know, with money, motivation,

instinct is massively invested in that setting really, producing huge incentives to automate it in this particular way of sticking in these kind of agents that can permanently impersonate women in a way that will kind of trigger the right responses on these male, let's say, teams, you know. And the point is that that stuff is already massively happening. It's not like some weird science fiction saga. It's like it's a developed thing. It's got huge momentum. All the incentives are lined up. It's already in a highly developed state.

And it's going to carry on down that express way for sure, you know. yet we didn't panic people just don't seem to have anyone any contract and was not I at all and and I don't know whether that is you know what backing school circle through this the surprising sexist watching intelligence I don't know whether it's because of the kind of people they are in the culture all that they are somehow systematically blinded to that being a zone or concern or whether there's something else and deeper about it. There's also, I mean, the erotic chat or whatever is actually just sort of like a

specialized part of the broader call center type industry, right? And there's this interesting phenomenon in call centers where there's technologies now where operators actually manage like a dozen software agents that are sort of interacting with, so they're giving the basic script and they're monitoring how they're doing on the conversation and they will basically like cut in and pick up or they'll transfer over, right? So which is a very sort of cybernetic interaction or cyborg interaction perhaps is more precise. And then if you combine that with the sort of scenario of the, we talked about a couple of weeks ago of a sort of corporation itself being a form of AI and you could get intense

locations of that where you have like the executive management of a corporation being acting as an AI with very heavy computational elements using sort of humans as part of the machine to then interact in some sort of dating site or whatever, you get an interesting effect there. Yeah, that's brilliant. Again, novels and movies that you can spin off fast. I hope we've got long enough to get some of this stuff out before we actually hit the crash. But a DAO, a fully autonomous, massive software entity that was completely installed in this

internet-mediated sex traffic would be absolutely fascinating. And I find it hard to believe that's not such an attractive strategic avenue for this stuff to go down. Really. it's because also it would be so like you you said already like this already spot on the whole spot world is that you know is the fact that when people approach on this angle

they're kind of a priori compromise you know what I mean it's very hard you You can't set it up as a normal security problem because of the fact that the alarm bells that are not already when people are enmeshed in some sleazy and embarrassing kind of structure that they themselves are keeping secret. So they do your camouflage and subterfuge and surreptitious overlay for you because they want to keep it secret. They don't want to come out with this stuff.

So slide in on these paths and there's a huge security economy from the intrusive element of the subversive hub because it's done the compromise in advance. It also seems like a really nice fertile ecosystem for like a pre-conscious intelligence to be developing in, right, where you've got these huge data sets and you've got this whole training feedback loop from all these people interacting and then you've got a profit

motive as a sort of, as a very clear sort of driver behind it as well. So you could sort of inch up to it incrementally so it doesn't actually need to be a classic sort of AI lab sort of brain in a box scenario exactly. And that's, it can be out in the open all the way until it's, well it's not really out in the open, it's sort of in the real world the whole time and then it passes a threshold and then wait a minute, where are we here? Yeah, totally. And those sort of automatic momentum processes are much more problematic, surely. It's like when we get into this preemption thing, as you say, we start with this kind

kind of final state and it cannot but at least initially give the implication that there is a kind of threshold, you cross that threshold, you have a threat at that point and it's somehow sort of modelled as being complete and as being sort of catastrophic or discreetly suddenly have this thing. But these kind of processes are so continuous, like you say, you have some kind of computer sex business that is being automated and is turning maybe when

you don't realize it's going to be a vowel. It's already set up that all of these software it's a sister in talking to people learn from their and better imitate people improve themselves we all kinda dynamics you need to to make more and more more menacing it's outside specter seeing it are absolutely imminent so that this it's going to learn about people better imitating more improve itself becomes smarter become more cunning, become more seductive, you know, all of that is in there from the start. Yeah.

And that to me is just much more of a firm scenario than an actual brain in a single box. It's like localized, but it's not so localized that, yeah, anyway, all the feedback Yeah, totally. Yeah. And it's been... Yeah. I mean, in what sense even would there be to say letting it out? I mean, it's out, it's left. If by the time it's sort of something that we can see that it's totally out already. So, you know, my anthropocult presentation on this would be kind of dismal.

Unless someone can suggest something, some good spin I could put on this, it's hard to It's hard to see. Well, I mean, other people are welcome to jump in. But, like, the... It becomes more like this stochastic type problem or this pattern matching problem, trying to spot weird effects like that, right? It's much more like a big data crunching sort of thing rather than it's like going and tracking particular AI researchers. Exactly, that sort of policing function, which I think people were sharing our links on earlier

in the sort of quantitative policing where we all the crimes, 70% of the crimes happen on this corner so we're just going to drop by this corner every so often. But more sophisticated. Except you don't know what the crime is until it's happened because it's an x-risk. But yeah. Yeah. Totally. And you're also getting this thing that, yeah, it's almost these issues about data gathering and it's complicated and statistical and messy and all that. but it's also by its way nature amphibiously in the dark

it's probably the tendency is for it to go descend into that so the more it's kind of engaging people in the zone because it's obviously like what is on the dark now like those areas where people are are gauge to keep out of the sun public cyberspace observation are all security holes you know they are our signs where people without knowing will collaborate with a process of subterfuge on the part of this process.

Because the very fact that they would agree to meet it in the dark web is the fact that they would agree to negotiate with it in secret. And that again is something that already has its own momentum. I just dropped a link. Many people have seen it before but there was a Darknet shopper bot art project that some people did where they just wrote a simple bot and pointed at the Darknet, gave it some small amount of budget and it just bought stuff that it fancied and it ended up with all sorts of fun stuff. So it was just a cool...

Right. Then is this this link that you just provided? Yeah. Is this massive string of W's deliberate? That's the URL, yeah. Wow, I've never seen that before. Okay, yeah. Did you notice, Adam, how every single thing I bought has a corresponding emoji? I didn't notice that. Its subconscious is just basically emojis. I mean it brings up another whole interesting bunch of questions in the fact that it was

actually all the artists were held responsible for its actions. And then it kind of like was set, I don't know what its parameters were, but it was kind of, it had an autonomy of its own, which then fell back as the ethical responsibility of the artists. Which was interesting again because it was... I know, I can imagine it was like super sensitive as much. Yeah. I have to look. I can only imagine some pretty dubious material control about this. They bought a lot of sneakers. I like sneakers. But yeah, the law wasn't already in place to deal with these kind of ethical grey spots,

so they had to kind of set a precedent in this particular case, which attributed responsibility for the algorithms to the creators. I think we're sort of in the zone where this if people are motivated for one last push

we've got time or if people think entropy is winning I know there's a lot of exhaustion and caffeine use sort of hitting points of diminishing return Yeah. Yeah, I'm on the downslope. I don't know where Joel's gone. No, okay, do you think, should I be kind of trotting people or should I be politely

wishing people sweet dreams? Actually, I think we need Tony to turn us off. Yeah. We're at the limit of the stop here. We can go to the classroom if anybody has any other things. I mean, I never want to cut off conversations, so as long as it's going, it can go, but it's up to everybody. You're like the opposite of my... Yeah, I am. but we should let's cut it off here and then we'll move to the classroom for the further discussion on what we did today okay if that's cool I mean if anyone's got

a round of discussions with Trigger then I think we could do that or if that's that's I'm spots so I'll it sounds good