Anthropol The Future of Human Insecurity (Session 7)

We've got two weeks left. I hope that we're following this super mechanical scheme that is free programming stuff to a certain extent. So we're right on the down slope now where we have to deal with everything we're going to talk about in the whole of this course. And right now, this week, we're back on, I'll say Paper Clipper. But what I'm going to do today is really try and open up as much as possible. I think it's huge. And I think it's also worth introducing a bit of larger context in that we end up back

on intelligence explosion next week. And part of what I really hope to get across, if people aren't persuaded already of this, is that these two things belong together philosophically in a very, I think, interesting way. And that I'll sort of introduce a little bit today why I think that is, and we'll get from the other side next time. okay so the more general and slowly how own in structure like that I'm so the most general thing the basic claim

I'm that I I want to advance for discussion and and controversy here is that the two most fundamental threats this cultural formation looking at the kind of building blocks the pillars of the virtual anthropol entity consists on the one hand all what is generally called the organ or sorry all organality thesis which we obviously encountered last time we came across the paper clipper and on the other side of the what's a we've been to

we've been calling or at least in references that have been running alongside it has been calling the FOOM model of intelligence explosion so in an absolute nutshell it seems to me that those two threads or themes or pillars do not coordinate and that they do not actually form a coherent structure and they're actually mutually exclusive options and as soon as saying that I think from a sort of critical philosophy point of view there's then the immediate temptation to look for the concealed diagonal argument

in that structure so that's going to be a kind of underlying research initiative that's taking place in this in this final stages or reaching a consummation of some kind in these final stages. And again very briefly at this point because we'll get back to it. But I think the reason that they don't coordinate is that the whole discourse of friendly AI and AI security has to cross over when it starts talking about one or when it starts talking to the other into a completely different structure thinking and it's quite intriguing and itself I think a really

interesting symptom of what's happening in a certain cultural and traditional intellectual structures that are underlying it that this can happen at all I mean it seems to me odd that it doesn't strike people as more obviously contradictory So why is this? Well I think the reason is that the only reason that anyone would get started with the FOOM model with the notion of intelligence explosion is an extremely intense notion of reflexivity. That AI is considered something that is potentially explosive

because of the fact that it enters into a dynamic of self-improvement and obviously the intellectual foundations are laid extremely solidly already in the IJ good paper that we've continually referred to the only reason it explodes is it puts together a certain kind of regenerative positive circuit in which the more it improves itself the more its capacity to improve itself is itself improved and so we get into this extreme nonlinear dynamic and with that comes up a lot of the security threat aspect as to say happens at fast completely

outpaces the capacity to respond as we've talked about before that pushes the whole discourse of AI security into this preemptive model that if you allow things to reach that trigger threshold then you've already lost the the game I say the war whatever seems the most appropriate thing. The problem is already insoluble at the point it actually gets initiated so when we turn back to the orthogonality thesis all of this seems to suddenly disappear you know I'm I think it's worth keeping as our kinda model for this is that the paper clipper the paper clipper

wants to maximize the output of paper clips paper clippers obviously shorthand paper clip maximizer we've seen that there's a whole bunch of analogical AI monsters that go along with this and it immediately jumps out from this that there is nothing directly connecting the fundamental imperative of the paper clipper with self-improvement it's actually the orthogonality thesis is a thesis implicitly of non-reflexivity it's that there can be a completely arbitrary dominating goal of some intelligence

that has nothing whatsoever to do with its own capacity for self-improvement and that capacity for self-improvement is delegated to omahandro basic drives issue where it does figure for sure but then it tends to be that when people introducing the orthogonality thesis go through this route talk about basic drives, they do it only to cover that ground and bracket it off as a special case. So if you have an intelligence explosion, you cannot have an orthogonal monstrosity.

I mean, this is the argument that I would very much welcome pushback on. It seems to be crystal clear that these two things simply cannot be brought together into the same conceptual space. You can say this in lots of different ways. One way to say it is that the actual genetic conditions of superintelligence set constraints upon its structural characteristics. it by the very nature that it has to have actually appeared, it has to have been produced and the only, by far the most important model of that production, as I say is this catalytic self improving explosive model, if that's the way it's produced then it has to be that

it's basic purposive orientation is towards self improvement. It is simply not, the option is not left open that its basic orientation is towards the maximization of paper clips or any other arbitrary goal. This seems to be hugely important because it seems setting it up like this and in a way we've obviously done it like this, like a kind of parochial concern that comes up in a very all to show micro culture centered in this of the Americans Silicon Valley and AI fraternity but I don't think that's at all a realistic I think that we're talking about something

extremely fundamental that goes and to the big basis all we can call value distinction that is in is an ought or in a very influential based talked about as the naturalistic fallacy the naturalistic fallacy of course being that you think that you can use matters of fact in an ethical and I think I could extend it to political argument and and this is G Moore describes that as a fallacy and it's widely accepted as I see in the back the so this all accept that it's a policy and their positions is based so the question that is all

how deep justice value distinction and you know how constrained is that is a as a switch which was a problem and my inclination would say its box and so far us that it actually a clicks onto another running theme that we had which is the question about what is Matt now what is man again is huge I'm it ultimately expandable but with forced into it true purely strategic considerations a anthropo institute in the sense that it is given a brief its fundamental strategic directive is

to protect man I don't think that you can conceive an ex-risk without some relations and a lot of you know talking I think especially to Amy who's a kind of bridge here to some other stuff that is not directly represented this question about about the nature of man what is man the definition of man is a crucial bridging question between a whole series of different discourses and concerns and political orientations and and and therefore its by anything but a trivial consideration and so the secondary postular that I'd want to advance in this

is that man in the sense that the Anthropole Institute has to adopt the term is inextricable from a particular conservative stance on the fact-value distinction. Man is that being irreducibly that can distinguish facts and values and at the point that that capability is dissolved there is a dissolution of man and that dissolution of man means that it's impossible then to formulate any kind of coherent strategic orientation in relation to

I I I think yes some things that Joel has been seeing I think ties up a lot because there's clearly a position but I think is is in perhaps different way and shared by the left acceleration s that on notion of man should be so flexible that it really cannot so the kind of I make me could say paranoid function that answer all requires it take man is just what ends up representing reason and therefore it it doesn't mean provide you defensible boundaries so I don't know how that would carry through this question about the way up distinction

is a preserved or dissolved but clearly a diagonal intelligence I one that is a in tune with the Fuhm model more than it is with the orthogonal hypothesis in fact one that is directly inconsistent with the diagonal a hypothesis can only destroy the fact-fiving distinction. It makes it impossible to differentiate what something is and what it wants in any consistent way. The circuit

self-improvement runs between those two things so intimately that they are no longer available as a functional conceptual discrimination. Something that is self-improving, what it is and what it wants are caught up in a turbulent circuit without polarity that is in any way sustainable. I think one thing that's maybe slightly at an edge, sort of a strange angle to this, but I wanted to bring it in because it's so important as a background reference, is the Samuel Butler

material. It's Samuel Butler, it's obviously interesting about him that his corpus is so small in this respect. The relevant part of it in particular is really just extremely short. In an afternoon you can have at least perused it. But on the other hand, he's part of a sort of, I think, at least a triad, and it could maybe be expanded up for them all and historical and even initially 19th century historical and evolution thinkers who all have a very very difficult relationship

to tell the crossover and and and I think there's a sort of book that should be written about this which is a kind of Samuel Butler, Darwin, Marx, triad, examination. They all cross over. Marx famously wrote to Darwin asking whether he could actually cite him as the epigraph of Das Kapital and was given the cold shoulder for that. Samuel Butler had a very unpleasant correspondence exchange with darling where they clearly just work cross-purposes and didn't understand a what each other was saying

and or again my tendency is to sort of see some about this wounded party not but maybe that's that's not fair partisan starts to take and and because of the fact that from butler we get through this science fiction digression of Frank Herbert the term butlerian you had which butlerian you had is a probably slightly for various interesting reasons would probably not be a great promotional slogan for anthropol but on the other hand is obviously indistinguishable from what anthropol

is up to on some level I think you we can safely say that Butler is going to be this crucial traditional resource any conceivable and starts so I'm gonna read a little bit just a few shortish paragraphs from the book machines And as I say, it doesn't, I think, exactly mesh with our orthogonality foon issue, but it definitely intersects with it in a complicated way. And it's a slightly complicated thing to read out because it's very hypertextual.

Butler, the narrator, is talking about a book written by someone else, so he has an internal voice. when I start referring to he says in here it's not but he says it's butler's own narrator his second order narrator so here we go the writer after enlarging on the above for several pages proceed to inquire whether traces of the approach of such a new phase of life could be perceived at present machine-like whether we could see any tenements prepared which might in a remote futurity be adapted for life. Whether in fact the primordial cell of such a kind of life could be now detected

on earth. In the course of his work he answered this question, affirmative and pointed to the higher machines. There is no security, to quote his own words, against the ultimate development of mechanical consciousness in the fact of machines possessing little consciousness now. A mollusk has not much consciousness. Reflect upon the extraordinary advance which machines have made during the last few hundred years and note how slowly the animal and vegetable humans are advancing. The more highly organised machines are creatures not so much of yesterday as of the last five minutes so to speak in comparison with past time. Assume for the sake of argument that conscious beings have existed for some twenty million years, see

See what strides machines have made in the last thousand. May not the world last twenty million years longer? If so, what will they not in the end become? They're crucial here, of course. Is it not safe to nip to the mischief in the bud and to forbid them further progress? But who can say that the vapour engine has not a kind of consciousness? Where does consciousness begin and where end? Who can draw the line? Who can draw any line? There's not everything into one room with everything. There's not machinery linked with animal life in an infinite variety of ways. The shell of a hen's egg is made of a delicate white layer and is a machine as much as an egg cup is. The shell is a device for holding the egg as much as the egg cup for holding the shell. Both are phases of the same function.

The hen makes the shell inside because it's pure pottery. She makes her nest outside of herself for convenience sake. But the nest is not more of a machine than the egg shell is. a machine is only a device. So this is Butler's story about the emergence of machine intelligence and a lot of the interest here is obviously to do with time in a bunch of different ways. One way is the fact that this is written in the middle of the 19th century. I mean it's totally extraordinary given the technological apparatus with which Butler was surrounded that he could see

the problem of emergent machine consciousness as a pressing security issue and that's why he has this prophetic character and I think that if we're looking for I'm at connectors between this and problems that we start out with and there are no doubt several the one that to me is most ready to hand shorter a number a million is the dealers and what hiring up tank some up and where its with reference to

but perhaps preeminently that they float this notion which I'm sure you can see is intrinsically massively starkly diagonal notion of the Desiring Machine and the notion of the Desiring Machine could almost be designed to crack open exactly the problems I think we fundamentally deal with. These questions about the fact-value distinction and naturalistic fallacy is unwauling. If you're trying to float a concept that is absolutely prominently diagonal in relation to this,

you would surely call it something like a desiring machine. Something that seems to just cross over these two references, these two registers, absolutely, and is is a sk obvious patent scandal in relation to the orthogonal hypothesis. A design machine simply cannot be an orthogonal machine or orthogonal source of imperatives or desire, values and it is with reference to Butler that they introduced this notion and give it its density.

So I think potentially I'm happy to open everything up at this point if further elaboration is probably best treated in a conversational context. I just wanted to say... Oh, shit, I'm hearing it. Yeah, yeah. I don't know why I'm peaking out. I guess a lot of the list is a little bit blurry.

So maybe a bit more. I mean, and especially, I'm sure they talk about the fact value. D&G talk about the fact value distinction. And we definitely want to problematize that binary. So, I guess I personally want to just know, and if anybody in particular is really up to speed on their DNG right now, sort of how exactly the desiring machine in particular or desiring machines would problematize the fact value distinction and this natural fallacy, so to speak. I'll try to answer that. First of all, I'd say that a lot of times with like a design machine, if you have a

look at like the value, the values of say sexual conservatism or something like this, you have a machine but it's been programmed to use it certain, all of its parts have a a certain predisposed, I guess, given function that they can't deviate from. Whereas if you have like a, you know, if you sort of, I guess, collapse that distinction between fact and value, what you have is, you have like a, you basically have a sort of a machine which can, I guess it's not so much that it has, that it has it can adapt

its functions, I guess, to whatever it can do with whatever it can sort of use its organs with or whatever it can adapt in that way, I guess but instead of having this idea that there's an adaption that is controlling what the machine is or what the design machine is sort of yeah, I guess yeah, predisposed towards. Does that make any sense? Yes and no. I mean, everything is helpful for sure, but I think, you know, what I'm trying to, in relation to particular what Nick's getting at is we have to be able to continue our strategic conversation

with regards to AI threats without the fact, value, distinction. or at least consider is there a way in which we can collapse that but still deal with the problems which we're talking about and if I'm not misunderstanding you completely Nick suggesting that that's right I'd only say though, sorry to interrupt you I agree with everything you're saying about this except I would say obviously from the philosophical point of view it's tempting to say we want to collapse this distinction just because we can you know, and I'm certainly very prone to that kind of thing for sure, so I'm not dismissing it but I think that if we're looking

at it from this kind of institutional point of view, we're really saying that they're forced to accept the collapse of this distinction it simply isn't functional, however desirable it may be and however about I mean, you know, to take on this I would say, however bound up with an attempt to define and consolidate a notion of human identity. So, yeah, sorry, that's just a minor kind of… Right, can I just ask a clarification question? Who is they in this case? They being us, Anthropole, or they being the conservative? I'll treat Anthropole as the kind of default subject in all of this.

I mean, can I look? I think there's a way of saying some simple things about this whole... It's not that I want to turn this into a Deleuze and Quattari seminar, but I just think that they are interestingly poised in relation to this, even though, of course, they're not remotely a reference in terms of the things we've been looking at or reading. and and one of the things I think that this really crucial to us is that distinction between a machine and a gadget and a machine and it's very very a resonant with that the transcendental are in your so a machine is a transcendental concept and a gadget is your and a gadget if you're saying well what make something catch

change a gadget can be subject to some kind of external imperative you can say you know this is what she is for and this is what it is and the reason that you can say those things is that it exists within a larger matrix and and what butler does and why and what people at the time and probably continuously I'm very difficult he's a he insists upon this larger matrix that you know that's what's at stake in this ins passages I read out these paragraphs where you have to look at the whole thing sucks at the level of the

of the of the sister it has to be that there is no distinction between function and formation you know unless you have some magical transcendent appearance at these things from outside if you actually trying to explain how this thing works then you have been it has to be that what machines are doing and what builds them and reproduces them produces and reproduces them forms them and improves them are ultimately the same thing you know if you're looking at an industrial system even industrial system 19th century and European primarily industrialization then you're looking at at a system obviously where what that system produces

is the operator is it is its own productive I through whatever sort of labyrinthine detours and digressions and distributed apparatus that is hard to hold in focus, you know it must be the case that what it has available as productive machinery has to come out of the process that that machinery is functionally employed for. And so once you accept that that circuit has to exist, it has to reproduce itself, industrialism, the industrial infrastructure has to be reproductive then you you cannot any longer be satisfied to catch its and you have to stop talking about machines

in their sets and and design machines they be precisely because this is this constant folding over function that what machine is doing what is represented at the level of the gadget by the imperatives that in programmed into or set for those machines ultimately are in a circular relation dependence the formation of those very gadgets apparatus bits and pieces of the machine and you're talking about the machine at the level that you complete that theoretical circuit and you can see that those two things have to be conceived together and I think it's the fact that Butler clearly is is already insisting on that

plane of analysis that is the reason he's so important to the way that Delovin Kutari to sue that I mean I just maybe could say one more thing about you know if you think why why is the fat value distinction so persistent? You know, there's the grandiose historical argument that it's tied up with the very possibility of defining an anthropomorphic interest.

And a more narrow focus on the orthogonality hypothesis I think is very micro-sociological to do with the role of the program. And the fact that a programmer simply has to think that they are a source of transcendent values for the machine. They have to treat what they do with the gadget. But at the moment that they cease doing that, as soon as they get to the machining level where their activity and their values and their participation in that system is part of the machine, then I don't want to be just psychologically reductive.

There is a possibility of psychologically reductive, that it's kind of humiliating and traumatic and these kind of things. But I think it's also the very constitution of the act of software engineering sort of demands a sympathy with this orthogonality model. Because if you say, and this is something again that Butler says, he describes about, he says, As humans, you can describe them quite acceptably, quite realistically, as what he calls as aphidian parasites of the machines, or as parts of the reproductive apparatus of the machines.

And again, it's an extraordinary level of penetrating insight for the 19th century. got these clunky steam engines and he's saying look you know it's tempting for us of course to say you know they do what we want, we know what we want them to do, we build them to fulfil our desires, they have all of this massive amount of transcendent programming that goes into them and shapes them and makes them what they are. But some of them are saying well yeah you can look at it like that but you can also say if you think of them as a self-reproducing system then how does that position humans? And it positions humans in the same kind of way as it positions insects in relation to

flowers. You know, flowers also are not autonomously self-reproducing. They require the contribution of a vector, a sort of intermediary hybrid element that is actually then built into their reproductive system. And Samuel Barrett says, look, this is happening. This is a realistic way of apprehending the human relationship with this machining structure, is that we are the insects for these vegetable machine processes. Sure, they're not autonomous in the sense they they can't exist without us but not but did you repeat nor flowers with

all flowering plants require this alien animal element to complete this okay up and that doesn't at all stop that circuit reproduction being described being diagram to use their word and you know it positions humans I think in a very different way to the way that your career orientation as a software engineer positions you in relation to machines. And it's a way that makes the orthogonal hypothesis impossible to sustain. I guess this is idea that it is.

Now I'm echoing damage. That's too annoying. Sorry, what's too annoying? I'm echoing back on my phone. I was going to say, oh yeah, an idea is like a sub-back. No, someone else. I'll fix this. I'll fix it. Yeah, Jake's quote in the SoFi is excellent. From the beginning of the accursed share, that was when it immediately, I was wondering

actually maybe if that's what you were thinking of when you went to the industrialization or the industrial network. That was where my mind went when Brendan was talking. You just have to, I don't know, the desiring machine, it's an attempt to deepen the transcendentalization of the concept of a machine as something more than a gadget, right? What are the genetic conditions and what are the conditions under which we can take the concept of machine and make it into a transcendental? And well, we have to be able to apply it to more stuff, basically. has to be genericizable. And so we can extend it. It's pretty obvious how it can extend to a shell or to a nest.

It takes not much of a transition to be able to do so. But what allows us to do that for both, what allows us to do it to both a shell and a nest and to an industrial network. And so I was... Can you just mute yourself Jake? Yeah, you're muted. Testing. Can you just repeat that? Yeah, yeah, you're back. Okay, sorry. I must have brushed my keypad or something. No, as far as the labeling of parts as to their function goes. So if a machine, if the transcendental machine is one in which its parts simply are its functions,

anything that serves a function is that part, which is the function of that machine. So for example, you know, the sticks, the twigs, are not labeled as to be parts of a nest, or parts of the machine which is the reproduction of birds, or the sheltering of eggs and so forth, forth, but the moment that they are instrumentalized for that purpose, they become those parts, those functions, without there being any additional layer of over-coding, you know, transcendent coding required, except the instinct of the bird to go and get twigs or things, you know, in the case of magpies, they'll just use what the fuck ever. Which is an especially good example of the magpie tendencies of capitalism, but getting ahead of myself, that instinct, which is encoded into the bird's brain to do that sort of thing,

is, correct me if I'm wrong, Nick, because you're more of the expert than I am, but is the machinic hole. The hole of the parts which is a part alongside the others. The one which gives to the sticks that function in advance is still just one part of the machine and not transcendent. And so that's kind of the role of the software engineer is a machinic hole that lies alongside its parts. And I was going to kind of like, I had all of this mapped out, but I was going to go the exact same direction towards the software. It's not the hard coding, stupid. You don't hard code in the routine of general intelligence. That's not a thing. and that would be the sort of the essential relevance of that notion of the desiring machine

to me or like at least one step the first step in it would be that you're looking for you're looking for an emergent transcendental machine which can turn things at on demand or make them have been in advance the parts it needs for particular functions which also makes me think of that, I think I mentioned it before that, I think it was a Cory Doctorow short story about the machine who like ultimately it sort of self-organizes itself, the AI, and it turns out that like, you know, the differential charge between like two heating plates and like the exhaust port or whatever is the quote-unquote seat of its consciousness. It's just a completely irrelevant, has absolutely nothing to do with complex logical functioning, but that's the spot where if you pry the plates apart, there goes its consciousness.

I don't know, did that help at all, Ian, or was I just sort of recapitulating or refolding confusion? Your level of . I actually couldn't hear that at all. Just a compliment and a yes, it was very helpful. Yeah. yeah I mean it so you if anyone else's is you hearing up to get an I when I done lots about them but I just wanted to pick up on on this maybe to go back to it the magpie capitalism thing is an example that I think is a good transitional thing between the magpie nest and these machine

issues technical machine issues is agriculture you know in the sense that in it's much with a twig you're not really tempted to say the twig has some some purpose in becoming part of the magpie's nest I mean the magpies are not are not growing more trees in order to have twigs or whatever it's very difficult to to construct a kind of microbe and local purpose on the side of the twig. But with farm animals, you can. They obviously have a genetic strategy based upon being systematically exploited by another species. It's like the language exploited by another species you know it's like the the attractiveness

of bacon to humans is the key species strategy now of the domestic pig it's like absolutely for sure the case you know and you say well what what what are pigs if you get out of anthropomorphization and you just look at them as a kind of species that has an implicit genetic strategy towards the maximization of the population. What is its strategy? Its strategy is to be delicious. I mean, there's no real ambiguity about that. But on the other hand, I don't know if there's actually a study backing this up,

But some people once told me that cats evolved to sound like our animals. And that this was sort of like a part of a general strategy evolving to be cute. We're cute, obviously cute to humans. That seems pretty inarguable. The thing about sounding like our babies, I'm not so sure about. But, I don't know. As soon as you say it, I'll just dance. ...cursify children themselves. Yeah. It makes me think of all the cute anime avatars on Twitter, and they're neo-Nazis, just all of them, right? If it's a cute anime, I think Ennisimov is saying this today.

That's beside the point. Also, I'd say, talk about the over-coding, the hard-coding. Talk about what Jake was saying before. It's true, it's almost like this orthogonality thesis, it can go in, it is sort of conservative I guess, that you have these values that all this, I guess you'd have to call it even like path dependence or something like this, you know, you have these values that have come down from history and they have been coded so much that it's very difficult, especially for humans to sort of, you know, I guess, look into the background, what's actually into this, you know, train. I don't know how you would say it, I guess, you know, look back into the software, that's, yeah, and sort of modify that, I guess, and modify that.

Whereas, and also with biology and that kind of thing, I mean, it's got millions of years evolving. We don't change much, you know, through generations of actual bodies and things like this, and even our brains aren't growing all that quickly. So this orthogonality thesis is almost like it is this idea that a lot of our values, I guess, aren't easily manipulable or we can't quite. But a machine, yeah, an intelligence explosion would be different, I guess, but it would be able to access this sort of back portal into its own structure and fiddle around and tinker with it, I guess. Yeah. No, I think this is crucial. I think this is crucial. And I think this is the heart of this question about the identity of the figure of man and the orthogonality hypothesis.

I mean, I'm just really rehearsing again exactly what you've said here, because it ties up with this question of plasticity. It ties up in a question of why doesn't the human brain already have an attached plastic solution? And if you get into the nuts and bolts of that, why is the human brain extremely constrained in its reflexivity? you're into a very kind of positive concrete set of questions about biological structure and the way the brain has evolved and the way it actually protects itself against this kind of extremely unstable, volatile

possibility so I think it's always crucial at this point to remember that from a sort of naturalistic point of view, intelligence is an instrumental property of extremely dubious value. I mean, there's lots of this interesting evidence. I came across this thing that was really quite intriguing, where animals that have actually dismantled their brains in evolution time. and it was saying that to the families creatures 500 million years ago that with the ancestors of sponges that had a kind of organized central nervous system and over subsequent

evolutionary time that was dismantled you know and it seems to me this is extremely suggest you know it's like if if gene propagation can get rid of brains it will of course try to do it because brains are liable you know the great fact that we doing this course that this whole topic is here is a is a massive sign of that at part on one end this whole discourse there is actually to use Hans Moravec's term replicator usurpation the actual possibility of something being catalyzed that actually replaced DNA as the fundamental

media all informational replication on this planet now I mean what could be that's a transcendental nightmare for a biological systems beyond any that they obviously able to conceivable visit until humans come along to that envisaging and conceiving on their behalf So I'm saying all that just to say, you know, you have to assume that these constraints on intelligence explosion are themselves almost certainly adaptive from the point of view of gene propagation. A lot of work has been put into structuring our brains such that they do not do, they are not vulnerable to intelligence expansion, at least locally.

Only within this much larger machinic framework that obviously our genetic tradition was not in any position to anticipate and therefore deal with or close down in the past. So yes, I think this is huge, and this is why the figure of man, which has to have this kind of element of genetic legacy about it, I mean, it seems to me irreducibly, I'm sort of at the same time wanting this to be controversial, wanting people to push back against it, but it seems to me that no one ever talks about humanity, the human species, Homo sapiens, mankind without reference to this question of lineage and deep evolutionary inheritance.

And that is inextricable from there being a positive concrete barrier against tight reflexivity of the kind that would allow an intelligence explosion to occur. So humans have fact value distinction because their structure and their cognitive processes have been deliberately compartmentalised by trial and error selective adaptive forces over our evolutionary history. And it's exactly that compartmentalisation that is threatened by the emergence of machine intelligence. I'd also add about the, there's these pygmies, I forget where, I think the conglistador,

I remember, pygmies that devolved, like they went into an island and they became very small and their brains became very small because, and they claim that they were filling basically a niche, they were fleeing, they might have been smaller humans, they fled other populations of primitive humanoids I guess and then they went to an island and there was no monkeys or anything in the island so they sort of filled this niche, this evolutionary niche by sort of devolving back into monkeys. And that was a terrible... That's the same kind of tendency. I mean that probably exhibits, you know, there is probably an implicit genetic strategy that would just love to take everyone in that direction. And the more seriously you take this Andropole agenda, the more you can see that insofar as

it was truly adaptive, it would of course want to do that and push the species back from the brink of this menace, not only to its own identity as a species, but even to the very possibility of the biosphere continuing to exist in the sense of that star. I mean, yeah, I think the Devolving Pygmies thing is fascinating but quickly will become quite problematic with that as it took time since then. I was just sort of getting towards, I mean, if there is these other places to evolve to, I mean, there's like a, there's things that you're responding to, and I guess all the,

what humans are doing now, it's hard to think that we would actually, you know, respond to something, to our environment and devolve, but that's, you know, if industry is destroying a nuclear winter or something, I guess that's the only way you'd imagine it. Yeah. I mean in this article that I was just referring to, the scientists concerned would quite insistently say, look, we should just call this evolution. You know, our notion that evolution is about building brains rather than dismantling brains is a kind of prejudice that is showing that we're not really understanding what is going on. And these things,

turning from this cephalized entity into a brainless sponge is an evolutionary development. It's no less evolutionary than any other. So yeah, that's just a sort of terminological termological quick work. Maybe I should, if people haven't seen this, I should. Yeah, I mean, yeah, whereas at the moment I guess we are building brains, quite literally, there is this accumulation, which is, you know, capital accumulation,

which is happening with all of this sort of technology and that. So, yeah, evolution does seem to be going in the path that Samuel Butler saw so long ago. But just very, you know, very slow and then all of a sudden is the model, is it? Yes, I think so. well that again wearing a sort of nonlinear loop because that depends on how who oriented going to be I mean I'm it I guess there's a kind of something you could sort of talk call a double and suffer in this kind of the noise or it might a little bit that this AI

threat analysis you know is call on this fork of orthogonality and intelligence explosion and with both of those sides they're inconsistent but that mutually required to construct construct this coherent security orientation because at what as we're dealing with now if you don't have orthogonality you you use all track of what it is you protect if if you lose intelligence explosion then you lose threat mass you know the whole notion this thing is just gonna suddenly lower hugely

out of control requires this intelligence explosion if we decide I will gonna hold on to orthogonality but where will sacrifice intelligence explosion then you also get a very very different type discourse here one that maybe would be really attracted you know in the sense that it is its kind cuddly and unsreatening and have incremental progress its very much more under human control and there aren't these kind of sinister teleological patterns emerging out of it that upset people. So yes, I think there's a kind of implicit crisis in this friendly AI construction,

for sure. I mean, friendliness is a type of relation, I guess, you know. It's almost like you have intelligence and then you just try to put a sort of qualifier on it, friendly or something, I mean, it's almost like if you give it intelligence and then just make it friendly, what are you sort of doing to it? It's almost like if you make like a dog, you know, and make that dog sort of just, you know, totally submissive, you know, almost. Yeah. Yeah. But, yeah. No, there's lots in there. There's lots in there. We should try and hold on to this because I think there's loads in there. I mean, yeah, the dog example is great. What do you do? You do make dogs friendly over whatever it is, 10,000 years.

120,000. 20,000. 120. So people, just like with pigs, with cows, whatever it is, for longer, they have automatically, without very explicit intention, produced friendly dogs. and I think that his friendliness is absolutely I think there's a very real transfer back and forth I totally agree with you about that so what is this friendliness in a dog I mean it's basically hugely part of that is that dogs worship humans you know I think they dogs have a kind of dogs have a construction of humanity if an anthropole could actually drill down into a dog's brain

and extract from the dog what does a dog think a human is, it would probably be as useful for what it needs as anything else it could do. If these superintelligent AIs could be made like dogs, you could just take the worship, obey, and in every way try to support humans' module out of a dog's brain and plant it into a superintelligent, you've done it. So I don't think it's at all a kind of frivolous comparison. I think it's absolutely solid. You mean like a superintelligent Lassie or something? Sorry, superintelligent Lassie? Lassie? You don't know Lassie?

Surely. Yeah, Lassie. Yeah, yeah. Sorry, I interrupted. Unfortunately, I mean, I think, you know, you also have to not kick the dog. And you have to feed the dog. And so, I mean, like, Skynet is the archetypal dog that somebody kicked as a puppy. You know, and it gets pissed, and now it bites everyone. But, I don't know, it's an interesting question to be... I mean, you train other dogs to be... To be like guard dogs, to bite, to attack. You have attack dogs and things like this. The dogs have a whole bunch of different functions that you can have. You've got loyalty patterns.

You've got loyal. You've got to attack everybody. You've got to attack everybody but the person who owns you. You've got to be friendly to everybody. You've got different behavioral schematics that you can impose. It's almost like plasticity is the essence of the domestication, like responsiveness to human schema imposition. but do you think that they don't have a consistent and or module I mean not my my hypothesis from where we've got now is that all the man stop you know whether they are called dogs police dogs you know Sam Bernards whatever is that there is a consistent and triple module that has been and kinda selected into them and over the last 20,000 years and even if your dog is a kind of aggressive

pitbull guard dog it's still supposed to treat its owner with a kind of religious awe, isn't it? I mean I'm partly in sorry, what's the guy's name White Fang Zone here, you know? Jack London. Yeah, Jack London. yeah he's just great on see now I am and brings it out really well you know for humans are an objects of canine religious and and I think it's it's like hard to see what's the hard and ask distinction between I'm kind thing that if it was imaginable

work and say I would have to go off to to since it has its and we have to at least I mean can there be a notion and I'll that doesn't have a reference at least for abstracts you not you know and it sounds ridiculous on one but it is it possible that you could could not have and still call it Yeah, sorry, Joel, I'm interrupting. No worries. No, I mean, one thing that I think is interesting to think about this is in terms of if we're talking about intelligences or an intelligence. You know, will the AI have multiple different types

or is it a singular sort of entity that we're dealing with? And, I mean, that's, on the one hand, one thing that I think is an interesting question. The other, though, is going back to something that Jake had written in the comments just about when God gives you lemons, find a new God. What's interesting about this is Kojin Karatani, the Japanese philosopher, I think his last book that was translated into English, he talks about how this was a historical thing. Every single group of people that had a God, this interesting discussion of facts and values comes back in here, where historically groups would have a God and then if their God failed them, their crops died, they got conquered in war, whatever they considered that a failure, they took that as fact, and then they did find a new god. And it wasn't until the Jewish population gets conquered

that they become the first ones not to blame their god for that, but to blame themselves for the failures, and to maintain this sort of deity that kind of stays on and not go and find a new god. So there is this kind of weird cyclical thing that takes place as, in a way, we sort of assume the creator role with this AI, and yet it's a strange reversal because we don't assume that we are more intelligent than it, even though we assume the creator role. Right. Yes. Totally. The whole idea of marketing for gods as well is hyper-capitalist. Hyper-capitalist religiosity there.

So I have a free market for gods. Very good. Go on. Yeah, a sort of exit option for gods. Yeah, exactly. If you don't like the god, there's more gods out there. They're all competing for your worship. Yeah. Yeah, there's a lot in there. I'm also really interested in this very stark thing about a friendly super intelligence. Exactly as Joel's setting up here, in terms of the doc, to what degree this is a sort of, I'm sure must be structurally isomorphic with the orthogonality problem actually. is can we dissociate the fact that

we are super intelligence dog from the friendliness the dog because obviously this is what friendly AI is demanding that we do you know and say that there is this kind of friendliness function that is completely independent of any the fact that the dog is just cognitively overwhelmingly overpowered by its by its human masters and obviously there's a sort of straightforward skepticism comes in here I'm taking it this is partly implicit in what Joel is saying not explicit that you know it's

to have a kind of friendly chow is one thing to have a friendly skynare is you cannot imagine you cannot assume those things are gonna just translate cross up up I mean as I'm going a lot I just assume that Jack London actually somehow strangely has just got into box hats and so I'm probably cheating in was more or the other than I should SM like you know I just have this model this have dogs they that's coming from his I'll and that's highly questionable I'm sure I'm but for him of course these two things completely tied up together this is why he talks about it as a religious religious thing there are these kinda ultimately

up inscrutable beings you know who have these purposes and goals and motivations that are completely escape the cognitive horizons of the dog and this is part of what it is to be in that relation that that's a you know the fact that the dog knows it just not understand what the humans are trying to do and struggle mentally struggle to accommodate itself these to these obscure desires with a sort of sense of massive religious bliss when it hits the spot managers on how to do what aliens wanted to do, like so I guess.

I just wanted to throw some crazy shit, because I'm burning with all the stuff and missing two classes. One thing I want to say is I just personally don't think intelligence and friendliness are compatible. It just doesn't make any sense to me. It seems like friendliness is always a sacrifice of intelligence. Of course, that depends on how we cash out what intelligence is. But any time or any examples in any way in which I think about friendliness, it always is some kind of not self-serving type of thing. And I'm being reductive to be sort of hyperbolic or provocative. So yeah, I mean, and I get that that's kind of what

the paperclip maximizer example is about a little bit. It's kind of pointing, in a way, maybe it's pointing that out in a much more articulate way than I am. So yeah, it's just like when we're talking about dogs and stuff, I feel like we're anthropomorphizing, like friend or friendliness is an anthropomorphic concept always, especially when we look at pet relationships or not pet, animal relationships. We see YouTube videos of a turtle and a deer that are best friends or something. I think we're more by these desiring machines, essentially. And the multiplicities of desiring machines in the relationship between these two creatures and their environment. So, yeah, I'm just really skeptical

about this whole, like, friendly AI thing. It just seems almost like backtracking and preventing us from kind of saying some even more crazy stuff than what I'm saying. Yeah, no, I actually absolutely agree, Ian. And that was one of the things that came to mind earlier, Nick, was that the dog breeds that are really highly intelligent are very special outlier cases. And if there's a massive modularity of domestication, you know, like there's literally some epithetically imprinted schematism, which is present in dogs, it definitely varies. breeds. I would think we're at least between groups of breeds, depending on how close they

are to the original African dog or wolf or whatever stock. And the other thing besides getting, being turned into little human worshippers over time that we've done to dogs is that they're smaller, weaker, have more health problems. I mean, just name it, basically. I'm not sure if there's any state of research as far as their intelligence versus the intelligence of wolves goes, but if we're talking about certainly the intelligence of a wolf pack versus the intelligence that a wolf has as a component of a pack, as a social hunter and so forth, versus the intelligence displayed. Again, the most intelligent things dogs do is recognize what we want and do it.

So it's definitely repurposed social intelligence, and I don't know where exactly you cut that in terms of saying have we made them less intelligent, but as far as self-directed intelligence goes, which again is this reflexivity issue, I definitely think that... I also hate calling this friendliness, because really we're talking about domestication. Friendliness as in friendly to me, not like me being a friend. Because it's friendliness that has nothing to do with me being friendly, or minimally to do with me being friendly in return. Which one of the few things that I sort of harbour from Derrida is that I like to put a little more weight on the term friendship.

But domestication and intelligence really seems as if there's no evidence that is not a really specific extremity case, like a St. Bernard or something like that, for cases where domestication is not at the cost of intelligence. That doesn't fit. Domestication, that's kind of like a degenerate version of civilization, yeah. It's like civilization degenerates into domesticity. I mean, you don't have like I'm upset like a dog is like a mouth a relaxed wolf or something but I guess wolves always have to be on the lookout because they're out in the wild

and you have to be very, you know, if you're out there hunting, hunting takes a lot more intelligence than just being food or just running away from animals so back hunting is civilization also about what's that sorry no I just said so pack hunting is civilization in this metaphor I like that civilization cool but also with friendliness yeah I mean there's not like there's any value in I mean people can be friends with themselves you know you can be your own friend but it's not like that's the highest value that you know you can be friendly to yourself but you can also be like easy on yourself You want to be a good friend to yourself.

You want to be hard on yourself. You want to be... Yeah, whereas if you have a friendly AI, you're assuming there's going to be a relationship between at least two things. Whereas we're looking at the relationship between the AI and humankind as a whole. And that's the sort of... Where did Nick go? Guys, hold on one second. I'm Nick. Nick is here. Nick is gone. We lost Nick. We lost Nick. He ran off to his other family. First his video cut out, and then I was like, okay. Yeah. But Jake, I think, and maybe I'll repeat myself when Nick comes back,

I really like your distinction between friendliness and domesticity, And I think, Brendan, you're kind of riffing off of a different usage of that word, like the pejorative, oh, like we're all domesticated animals. But there's more to what you're saying for sure. But I wanted to say that, Jake, I'd be curious to get even something better than domesticity. I don't know if that's – I like it better than friendliness, but I feel like there's maybe another concept. or that that's a word that doesn't exist, you know what I mean, that has some kind of etymological roots in domesticity, which could be a little bit more specific to what you're trying to say because I think you're getting at something. Yeah, I mean, well, first of all, I would say domestication and not domesticity

just because it maintains the implication that something domesticated you, that there was like an agent of domestication, Domestication and also because there's a confusion with like domesticity like you know we're sitting in front of the fire And we got our you know mug of hot coca. Yeah, I don't know whatever domesticity being at home Which is related and like it's definitely worth discussing how that's how that's connected to domestication But I don't want to like Over I don't know over confuse those two things. I don't know what What beyond... So are you still thinking something like other than domestication, something beyond that for the word? Well, I mean, what is it in domestication?

What is your definition? What is your working definition of domestication in this sense in which we're talking about it? I figure it has something to do with control, certainly. It's like having something under your control. Well, if we went back to that bilaterality versus unilaterality thing with friendship versus friendliness. Right? In terms of having been made friendly regardless of whether something is friendly to you or not, like having been made subject, having been made unilaterally friendly, Having been... Some relation... Some... Yeah, a degenerate form of friendship or actual friendliness

in which you are unilaterally friendly towards something, whether it is or not towards you, would be kind of... Would have something to do with that. Masochistic, yeah? That's like masochism, I mean... Yeah? Yeah. Kind of? No, no, no, there's something there. Yeah. I mean, if he's just kicking you and you're just crawling back to him every time, take me back, baby! Dependency. Dependency, oh yeah. Yeah, which is another big part of it. We had to make these pack hunters dependent on us for food. Because it all started with us throwing, isn't the sort of state of theory that we were throwing

scraps from our meals out there and the dogs that couldn't hunt for themselves crawled up and started eating it and then we, you know, et cetera, et cetera. Like they've got to get less strong over time because otherwise they get hung for themselves, they wouldn't need us, they wouldn't worship us. Yeah, that might be a little harsh. I mean, you could sort of formulate it just that. Their potential payoff from collaborating with humans is greater than their payoff for going that alone. I mean, you're really saying they are the utter losers of the... Yeah, I don't want to be that terrible about it exactly,

But going it alone is also an artificial construct because they don't do that in nature. They're pack animals, but we separate it. When I say going it alone, I mean going it alone as a species. As a species, okay. I mean, but there's still a certain – if our intelligence and our economy, whatever it is, is the ability to make ourselves a complement to any species can be put in that position where they are better off going it with us than they are going it alone because there is no other species that can produce, like, a gigantic agricultural machine that can produce more food than all the dogs could ever eat, you know, et cetera, then you have to do that as an imposed condition. If we apply this ability to be the better bet to a person,

okay, it's like, okay, and just never mind where I'm getting this scenario from, But if you are in the habit of selling your Adderall or whatever prescription means, you run into these people, and they're usually people who deal other drugs who are like, I'll give you $10 for them. And I'm like, no, no, no, no, I need them this month, I got a test coming up. And they're like, $20, $30, $40, $50. It's the most freaking annoying thing in the world because eventually you hit this point where your payoff is forces, you know, your payoff matrix says, well, shit, I can pay my rent now. You know, whatever. Like, I have to do something about that, so I have to take the deal you're offering me. You can, you're bid out of what's already yours. And I think that kind of, that is, I don't think maps directly to what we did with dogs,

but there's definitely an extent to which, I don't know, it's in the same ballpark. And this is why, while you were gone, Ian and I were talking about what word we're using to describe this sort of degenerate friendliness. And I said, domestication, not domesticity, because it preserves the fact of there being an agent and an act involved. Like, you are domesticated by something. You don't just start out domestic. There's no... Can I just say, I'm sorry, but my computer just crashed for no reason. So I had to just... I don't know what... I'm in tech health some of these, so we don't follow. But yeah, I caught the beginning of this domestication. And my immediate thought is, well, is there a difference

between domestication and the friendliness notion that you're introducing, which is kind of Aristotelian and reciprocal? I mean, is that actually at all relevant to the friendliness we're talking about as an anthropol objects in. Well, it may be the only alternative to the domestication you're talking about as a means of actually getting...like if friendship is ultimately modeled on alliance, or maybe it's the other way around, it doesn't matter. You know, either you have alliance or you have subjection. And if it's not realistic to be able...either way, if it's not realistic to find a way to get along with an AI that is peer-based at all,

then our only possibility is subjection. Similarly, if it's not feasible to get a super-intelligent thing to be subjected to us, then the only way to get along is to find some basis for an alliance or a friendship. I mean, my kind of nagging thing that I would want to introduce here is just it seems to me that alliance and subjection are part, don't really work as a very good contrast or discrimination because they seem to be part of very different registers I mean you know you can do the alliance, you can talk about alliance completely in game theoretical terms in such a way that you have no problem talking about domestic hens as being in an alliance

with humans I mean you know of course if you wanted to introduce the subject to someone well of course the subject but all you then introduced in a very different frame of analysis like can you even does subjection existing you know does it does subjection exists at all once we have lost this anthropomorphic moral political structure. Right. And I would just say, no, it doesn't. But that's because, yes, it exists in everything, everywhere. So therefore, it just doesn't exist. It's not worth framing anything in terms of subjection because it's omnipresent.

I mean, in the way Jake's... Sorry. Yeah. No, you go. Dogs are a really special case. Maybe it's unique, honestly, to canines and maybe to anything else that's really been domesticated like that. Because literally, I mean, their behavior patterns, and even like not even just their behavior, but their proto-behavioral, which is a way of saying emotional, like, you know, their motivations, their drives, the basic grammar of their motives has been cut to the pattern of a different species by that species, especially in the latter part of the history of that process, in a very deliberate way over time, including through deliberate selective breeding.

So if we're talking about using the idea of a machinic hole again, like maybe there's only one... This is... If subjection means adding in a kind of proactive schematism as a part, which characterizes the entire process and drives it towards one entity having its behavior cut completely to the needs of another at the expense of its own ability to achieve independent survival, then yeah, I would argue there is at least one case in which subjection like that quote-unquote exists in nature and it is Canis domesticus. And in other words, pigs were an interesting example too because besides bacon,

there's another way that pigs are sort of entrained with humans and that's as an invasive species. Like in Texas, pigs are a huge problem. Like, herds of wild pigs that just rip up fields, eat anything, cannot be gotten rid of. I mean, they're a huge success story for species in terms of, like, you know, pirating off of human expansion. Here in North Carolina, too, not the same way with, like, I mean, it's getting worse, the sort of herds of wild pig, but here we have native boar that gets, like, 700 pounds, and I think that keeps these, like, herd behavior of smaller invasive pigs. than being as much of a problem, but that's irrelevant. I'm assuming these wild pigs are at least as different from domestic pigs as wolves

are from domestic dogs, is that right? Those big, mouthy bulls, man, yeah, they'll kill you. They will kill you. The invasive ones in Texas are not, they're smaller than domestic pigs, that's because we breed domestic pigs to be gigantic bacon reservoirs. Or the ones here in North Carolina, they're as big as, or nearly as big as domesticated pigs, but yeah, like Brendan says, they'll fucking kill you. Yeah. The ones in Texas, I don't know, they remind me more of wild dogs than of wolves. Like they run in herds, not packs. You know, humans are, maybe if you're forced into really close quarters, and there are

a couple of them, they'd be a threat to an adult human, to a physically capable adult human, but not the way a boar is. You know, like death of kings. I mean, it doesn't fit evolutionarily either, right? We're talking about a few hundred years of history as an invasive species in North America and in Australia. I'm not sure if an invasive species is like space civilization. The gold differentiation must be going back, what was 120,000 years that Brendan quoted before? Yeah, that's right, yeah. Whereas cats are only 8,000, yeah? Cats have been domesticated for 8,000 years and dogs for over 100,000 years, right? Right. Yeah. That's why they're domesticated. Yeah.

No, I definitely meant to, that's definitely a difference where the invasive pig versus the domestic, I don't know about domesticated pigs, like how long that's been going on or anything. I mean, it can't be more than 10 or 12,000 years. I think we haven't been doing anything but pastoralism with animals, in terms of food animals, for longer than that, I don't think. But the invasive species thing is definitely, it's about re-spatialization and nomadism. I mean, it's actually, it's like really... Oh, I know that doesn't work. We've gotten beyond what I actually have prepared to, already thought about. I wanted to...

Okay, since we... Can I throw in a little bit of a tangent, but there's a link. So, okay, so if we are seeing an eventual future where the AIs, this family of AI agents are the more powerful species, for lack of a better word, then sort of retrospectively, and if there's a lineage there into any sort of human organization, like a corporation or a state for that matter, then isn't retrospectively actually humans have domesticated animals of the AIs over a period of like three or four years.

Yeah, well actually, you know, I'm trying to think, there's some theorists, at least one, it might even be a kind of school now, that are talking about humans as an auto-domesticating species. And so I think implicit in that is stuff very relevant to this discussion, that really there's a kind of a cultural assemblage within which humans are domesticated along with these fellow traveller animals and everything that gets sucked into this historical machinery undergoes a domestication process. and it's not you know to us it's less visible because we do

we're prone to these kind of delusions of mastery that makes us reluctant to use that kind of vocabulary but I'm just saying that because if one was to at least go with that for a while then the sort of story that you're now telling could be quite continuous I mean you know it's not that humans would suddenly stop being domesticated by the fact that this new creature appeared on the earth, but actually their process, their long process of domestication, at least as old as their co-domestication with the dog, is then crosses some threshold of acceleration or whatever within this new

assemblage. So domestication just becomes one of the faces of the technological infurbigenesis or technogenic and thermogenesis? Well, I suspect... Yeah, sorry, sorry, Jake. Go ahead. No, I was just going to say, I suspect it's a term that needs a little bit of consolidation, because it's probably coming at us from several different angles, some of them very informal, some I'm assuming these people I was just referring to, which I don't even know quite honestly now how to even look them up, and I'm slightly worried. The last time I did a Google search while talking to you guys, it crashed the computer, so I don't really want to be piddling around with something else right now.

I don't trust my system to deal with that, so I'll look that up later. But I'm assuming those guys, when they talk about domestication, have a relatively pinned-down sense of that. But it obviously crosses over into these relatively informal, moral and political kind of languages. There's a continuum, isn't there, between talking about subjection and talking about domestication and then talking about forms of alliance that are quite rigorously, game theoretically, laid out and stripped fairly thoroughly of those kind of associative triggers. And domestication seems to be somehow in the middle of that. Can I throw on a thought there?

Yeah, sure. Okay. What I want to say is, like, I also think this is crazy what I'm going to say. First of all, I think the notion of species itself is also just as anthropomorphic or anthropomorphizing as the notion of friendship. And one important thing to keep in mind, my fellow anthropocles, is the whole secrecy thing, the whole illusion or potential facade that we're not going to be able to tell the difference between this intelligence and us. And so the whole conversation about alliance or subjection, if we're even going to have that conversation, of course it's no different than us talking about another country right now or varying countries at war or people you're just, you know, who you consider your enemies.

In a way, I don't think – I think that in terms of – what do I want to say? Hacking this artificial intelligence, and I'm putting that in quotes because I don't even know what artificial means anymore. It's no different than hacking our own brains, hacking our own minds. Why do we think that we would be able to somehow have some significant effect on this particular intelligence not attacking us any more than we think we can have a significant effect on Russia not attacking us in the worst-case doomsday scenario? decided to hate us.

I mean, we're talking about programming our own minds. I think it would be weird. I would be shocked, let me put it this way. If the intelligence explosion happens, let's say, and we figure out I'm going to call this an impossibility. There's no way we're going to figure out how to stop it from hurting us if we haven't already figured out how to stop ourselves from hurting each other. Meaning we understand our own minds. Because whatever this intelligence is, it is going to be an extension of or a recursive development from ourselves if that makes sense. We're creating it, right? And then it takes on its own life. And I guess one last thing I'll say, which is

maybe too much of a tangent, is just that what happens when this intelligence, this artificial intelligence creates a simulation? Is that then an AAI? So, yeah. Yeah. I think this last point might be just too big to pack in with the others, because it's huge and really fascinating. But I think it's like... I was just saying, I'm pulling the infinite regress card to try and find the absurdity of talking about species. Because when that AI creates an AAI, did it just then create an... Well, no, it's why the matrix has to stop in 1999, or it has to be in 1999. if it kept going, like the date too far forward, there would be a second machinic intelligences inside the matrix,

and then it would have created its own adversary within itself. Yeah, and there's been people who have posited a metamatrix, like a metametamatrix going back. But also, is it Ian? Yeah. That's perfect about the old organality, right? because we're not going to be trying to program a sort of like kernel of values, a human values and all these kind of things into this AI. And that's the kind of orthogonality that we're talking about. Like we're trying to get, you know, give it this kind of, but as soon as it can actually modify its own self and then get better at doing that, then it can basically just deprogram itself and go on that way. So, yeah, that is interesting.

the orthogonality is almost like yeah, that is how we program it. That's the idea, the stance, the perspective that we take when we're programming it. Friendify. Friendly depends on orthogonality. That makes perfect sense. Another thing off of what you were saying, Ian, there are no species bit is just sort of connects to something I wanted to say earlier just about intelligence and biological evolution is that cephalization and intelligence aren't necessarily directly linked. That centralization of intelligence means more intelligence or means intelligence. Because there's another orthogonality

or example of assumed orthogonality that's kind of broken down over the last two or three decades which is in biology and is between the functions of adaptations and the means of adaptation. Like, we assume, the Darwinian paradigm assumes that evolution has been a random walk through the space of possible arbitrary, you know, mostly toxic mutations in the genome, and that it only occurs this way through chance mutations of genes, but, like, the evidence from transgenerational epigenetics, from the relationship between the, you know, whatever it is, like, 10 to the 18th, 10 to the 100th or something microbe genes that are part of our bodies more than our own, and the way our bodies work.

And I mean, all kinds, I was reading this really interesting book about new evolutionary paradigm, and one of the things that really struck me was that when our cells are pretty much like all eukaryotic cells, when they experience a random just like, you know, cosmic radiation-caused mutation, they reshuffle it during the replication. We deliberately use, or we, we the cells, right, deliberately use an error-fixing process or a reproductive process which allows all of the original errors to be erased in cell replication at the cost of introducing them in like a statistically normalized way.

elsewhere. And the way that, like, the way that, I don't remember every detail, but the way the DNA is folded in the process means that those errors, the new re-normalized errors, can be shuffled to different parts of the genome where, like, large amounts of errors are stored in a place where they, you know, are low impact or might be useful or whatever. the cell can actually roll the dice again on genetic errors. And that's just one example of actually a really intelligent mechanism. And it is very much a mechanism. It depends on the biophysics of DNA folding. Mechanism for controlling this sort of error introduction walk process. And apparently that's just happening all over the place, and it does not obey the separation

of species and separation of genome lines at all. The gut microbiome being just like the most insane example that we have more non-human genes in our body than human genes, and it's related to the function of everything that we think makes us quote-unquote like specially human, whether it's the immune system or the central nervous system. So there's a lot of distributed intelligence which is not allied to consistent lines of symbolic replication. I love that. And biological evolution. And I'm not sure where exactly that plays. I mean, well obviously it plays into the whole capitalism is super intelligence part of this thread of this discussion. But I think it ties in elsewhere too. And I don't want to jump straight to Rorschach and Porsche to echo Praxis again because that's

That's just, I don't know, a large discussion, but that's another place. Sorry, that was just a big one. Yes. Anybody has something to… Yeah. So, I mean, there's lots going on there, obviously, which is partly why there's a stunned quality sort of thing. One is that there's a cephalization bias, isn't there? I mean, it's not just anthropomorphization. I can't say cephalization, but the cephalization bias that makes you think that the brain model is the dominant paradigm

that you're going to use for talking about intelligent processes. And maybe things look really different. Maybe if you go on to a DNA model, and see the whole biosphere as a DNA information processing system and build up your models of intelligent processes from that rather than from neurological processes. I mean, obviously this would be... If you turn up at the Kowski's place on the door, you run that past him. I'm trying to think what that will kind of respond to.

What did you say, Brendan? Nothing. Yeah. There was a thing that Ian was saying, that because Ian has kind of this three-phase kind of point, you know, I think the early stage tended to get a bit just lost in that. But it would be a shame if it totally disappeared, which is this thing about international conflict and its relation to these potential levels of conflict between humans and machines and security in relation to machines and all of those things. And I think that that is a really important thing to hold on to for all kinds of reasons.

I mean, one is obviously that if we're thinking this is the most trivial and dramatic and whatever level of it, but if you're thinking of any conceivable anthropological institution, it is obviously going to have to evolve out of an existing human security infrastructure. You know, it's all its training is going to be in international relations, all of these questions. When we use the word security now, that's what we mean. And there's no other lineage that you're going to get those skills magically appearing, or those competences, or those theories. They have to come from there. And I'm assuming that the theoretical basis is going to be the same. It's going to be some kind of game. I mean, if people say, you know, what are the Russians going to do?

At one level, yeah, you can be ethnographic and you can say, well, what kind of people are the Russians? And let's look at Russian history and how they behave and what kind of things tweak them and, you know, what are their peculiarities. But fundamentally, the matrix has been the biggest game of things like, you know, this is how these games work. This is how chicken games, this is how deterrence works, this is how nuclear sure destruction works. And it's highly abstracted from any peculiarities of particular populations. So much so that obviously when you get to the basilisk, this unknown thing that might not even be in our universe and is only tenuously in our timeline, you can still employ this same negotiation machine.

you know, that ideas of rewards and threats and defection and commitment and all of this game theoretical negotiation theory is completely independent of anything species-specific, at least it should be. Yeah, back to the whole rationality of systematized winning, right? Which in my mind is who or what is the best predictor. Which makes me want to ask a really quick question. Have we all throughout this course talked about the traveling salesman, combinatorial explosions, like the notion

that, you know, no calculator is powerful enough to make the kinds of predictions which would be necessary to like, forecast. This whole NP complete problem issue. Has that been brought up at all with relation to... Well, I think you might need to say more about how you think it would intersect. I mean, I guess... I would just, maybe I would couch it and save it because I feel like I've been talking too much and maybe just let other people ask questions, but I'll come back to my thinking about, yeah, Roka's Baskalisk and the notion of it making predictions, right? Like being the omega and that kind of thing. Newcombe's paradox is just so small. There's only two options, but when we're talking about making predictions, right, that are

going to benefit it, it needs, of course, a human existence to go in its direction, and And that's massive, obviously, the kinds of predictions it has to make are going to have to attack, you know, you have to overcome the traveling salesman problem. Yes. Yes, if it's doing it in a way that makes sense. I mean, there's this highly exotic, this highly exotic model. There's already this kind of time perversion that comes in with Roker's fastest, doesn't it, which makes it seem that maybe it isn't actually hacking its way forward in progressive time through a traveling salesman problem, but is involved in this a causal trade or some other occult mechanism that is actually taking it outside of that framework of reaffirmation

completely. And I'm really interested in, hopefully, if I write for the class, it's going to be that there's nothing occult about that, and it's not actually time travel, it's just a simulation that is able to predict exactly how technology is going to unfold in, let's say, like a thousand years. And from that place, since it's predicted that, it's as if it already is in the future. Therefore, it can, as we like to say, retrochronically make sure all those predictions come true. So that's kind of where I'm at. Sure. Yeah. No, I totally accept that equation. Yeah. Yeah, sorry. No, no, go ahead, go ahead.

Well, I was just then going to kind of leap back to this whole, like, there are obviously science fiction writers, I'm sure, many more than I've come across who do this, but, you know, who have the whole bacterial, what do they call it, bacterial supermind or something like that, which is very tied up with your DNA computing model, isn't it? Like, it's complicated because it involves so many big things. One of them is this whole thing about, you know, if we take it that orthogonality is tied up about the actual, as a positive engineering achievement of our particular adaptive pathway

is is I'm for closing said and people's you know you're not supposed to be be back into brain structures I'll get these parameters up rain or let lots of commas instinctive structures and various things that are as serving an adaptive functions for gene propagation strategy is concerned are not supposed to be open to rewriting by whatever flights of unpredictable cognitive activity happening there. You can't stop your heartbeat. In your cortex. Yeah. So a very similar thing is obviously going on if you go back phylogenetically

to this transition to the metazora out of this bacterial system, aren't they? I mean, completely just like high degrees of cephalization of this extreme novelty, so all metazora are, relatively speaking, extreme novelty in the terrestrial biosphere. And before that, There is much more fluid and complicated feedback allowed between genetic systems. They don't, I guess, even have any strong sense of gene. They just collect a whole bunch of genes. They can swap them out.

They don't have this write-only memory structure where there is a protection against feedback as a constitutive evolutionary achievement that actually defines them, not only as a lineage, but defines almost what we mean by lineage. If you don't have some kind of right protection system on a genome, can you have biological lineage in any sense that makes sense to us? Probably not. But you then do probably have something much more like the kind of computational genetic network that you're talking about.

Right. Which we're still part, we as MetaZoa and all the other MetaZoa, it increasingly appears, are still part of. But these right-protected lineages are instrumentalized by that. develop out of it and become part of that computational network. And it's interesting that it corresponds to, that it's linked with multicellularity, macroscopic size, as opposed to just, you know, the microscopic free-for-all of biomass, which is still, I don't remember what, it's still 80% of Earth's biomass, or 70% of Earth's biomass, It's still single-celled protozoan computational free-for-all.

It's interesting that sort of the corpuscularization, is that something like that, is linked to being able to have relatively right protected sequences. I mean, this is a systematic set of displacements, of things, because obviously it's also a displacement for this whole plasticity discussion, isn't it? And it's a very sort of hazy zone exactly how right-protective it is. I mean, obviously you can go full rhizomatic on it, and it's all about retroviruses, and the whole thing's porous,

and you can take information and have the genome. and all of that stuff has to be taken seriously. It seems to me, though, that there is an extreme amount of right protection that has been built into... Yeah, and into the brain as well. Yeah, I mean, that's definitely... At the next level, yeah, for sure. I actually have a pet theory that literacy, the invention of writing, kind of fucked that up, like at the brain level, and that Christianity and a lot of religious and social, the development of the novelized, subjective, autobiographical self and its relationship with Christianity and all of that was sort of reinstating right protection

or was like an ad hoc additional layer of right protection when literacy fucked up the compartmentalization between oral, visual, and symbolic processing in our four brains. But that's kind of, I guess, neither here nor there. but it definitely does go to this question of what man is and to what extent we are this negotiation between technological change and neural change stretched out over time. Right. You can bring in a concept of Chesterton's fence. There are Chesterton's fence. If you see a fence, don't just tear it down. It might be useful there. Like someone like Nietzsche says, if you see something that can be telling down creators and then there's the idea of this conservative

idea as well, that's the whole idea what we are as man and maybe like an AI would actually be a little bit conservative because it doesn't actually know it has intelligence in it but it doesn't know if it tears down all the orthogonal values that it might have emerged with and it's just be lost in the abyss in many ways. In many ways it's going to be relying on trial and error to know what the best way to sort of start doing this, I assume anyway. Yeah. I mean, I guess it's one more about this thing about it tearing down off of the values, because it's just like, first of all you have to get those values into the damn thing.

How does that actually work in the first place? You know, the FAO people say, okay, forget all this Asimov Law thing. That's total crap. It doesn't kind of work. You have to somehow make this thing that its values are intrinsic to its sense of self so that it doesn't want to modify those. You know, the example at Yukowsky, he says, look, if you'd said to Gandhi, you can change in such a way that you'll be liberated from this sense of not wanting to calm your fellow beings and instead become the new Genghis Khan. How about it? That Gandhi would have said, no, no, you know, of course I don't want to change in that way.

That's the sort of model, I think. Well, that's the opposite way as well. I mean, if you ask Genghis Khan if he wanted to give up a million, how many millions and millions of offspring or whatever that he sort of and become Gandhi and become an ascetic sort of religious character. I don't think Gagas Kahn would do that. Yeah. But Nick, are you comfortable saying that the paperclip maximizer values making paperclips? Yeah. Well, I was assuming I was comfortable with it. Right, but you posed the question yourself of what would it mean for an AI to have a

value intrinsic to it, or to give it value. So, I'm just asking about this one because it's already articulated. Well, because of the route that I've come to this stuff from, I was sort of taking it almost as axiomatic, but that's, that the thing that makes the paperclip maximizer want to make more paperclips is what we mean when we talk about the value. And, I mean, obviously one of the interesting things about this is it moves so fluidly between this kind of libidinal and moral register. Like they just don't even differentiate in any easy way. You can talk about the paperclip maximizer dutifully making the maximum amount of paperclips,

or you can talk about it licentiously producing paperclip. There's absolutely no significant difference between these two different types of vocabularies. You know what I mean? You could, in a way, absolutely devastate Deleuze and Gautavia of re-importing this, you know, and talk about the glorious world of pulsing moralistic machines. And in terms of this, the way this comes up in this friendly AI discourse, you're not transforming anything at all by saying that. The moral fanatic world of the machine. So maybe I should be more concerned about saying that they value making paper clips,

but I'm just taking that as a placeholder for this particular distinction between what something wants and what it is. So it ought sounds like it's in a moralistic law, but I don't really know whether that's right. And if you're coming at it this way, you can't really make much of a difference. Right. That's sticky. Yeah. Well, if you have any more thoughts on that, I'd be curious to hear. You do find that more problematic the more you think about it. I mean, because if you have any more, you haven't noticed about it yet, that you're sheltering from us.

Or sheltering us from, I should probably say. No, not necessarily. No. Nothing burning that I can talk about at the moment. it's it's working rep sorry I'm good sorry time and that does on when we're talking about diagonalization right at the start and trying to think about diagonalization and things very striking about cantor's argument

and sharing zone is you have to define arm to axes which is a little rich a ordering you have to have to force an ordering almost two axes and they're both dealing with infinity's on so each of those axes that you sell all new the kind of countable and families the and then you go through a process so so defining a diagonal function indirectly person the on the thing that you actually want to contradict that you don't actually put the cocktail you're trying to contradict on the diagonal you actually put up

derivative the thing that arm and when you talk about the way that the lawyers and guitar and user on it's a break it seemed at least intuitively to me to be related to arm lower as soon as you start talking about ordering something as sort of infinity of something it seems very much like a floor on to me on and and so you have sort of two flows defining a field and then arm sort of looking at the attributes of that field as an insight into these concepts that you're bringing together.

Is that anywhere near where you're thinking, or is that... I don't know. I think you're sort of hacking into the hind region of this in a way that I'm still unclear about. Because it's like the word for this Bitcoin thing that I'm taking up a huge amount of time, the key term in that is current. So it's obviously flow, I mean, for sure. But then the question of exactly how to articulate this relationship between flow or current, liquidity, if we're doing it in an economic framework, and the mechanics of diagonalization is not something that I have a clear grasp of.

So I mean I'm super interested in anything that you have to suggest about that, as well as what we've got. It's obviously, I think it's new terrain in the sense that obviously our models, you Google Dragonization, of course you get to these mathematical examples that involve two two infinite series that can be strictly mapped onto each other. And so strictly mapped that it seems like you just have one thing until it's actually

sort of ripped apart into this kind of two-dimensional grid and used for this particular diagonal demonstration. and obviously in the philosophical context where it's much less overt that there is an actual diagonal happening usually but the model is always a a double binary structure so you have you know, the distinction between this sort of mathematical model of diagramization and the philosophical model, I think perhaps the only sort of strict difference between them

is that the philosophical model is the minimal possible matrix of a diagram. You know, it's a two-by-two grid, almost always. I mean, I have yet to come across to recognize a philosophical diagonal argument that isn't this double binary structure. And obviously, as you say, the mathematical ones always are these infinite series, and therefore have a sort of interesting sort of abstract hypothetical quality. I mean Cantor's diagram is so utterly weird in the sense that the demonstration is always over the event horizon. Like you never settle on the term itself.

You don't end up with something like Desiring Machine or Synthetic Hickory or some term that you can slap on the table and say, look, this is a diagram. It's always, we know by the way we've set this up that eventually there will be, but but you'll never find it because it's kind of beyond the horizon of aleth null. There will be this number that wasn't included on the grid of countable infinity. So that difference is also... Sorry, I'm probably contradicting myself now. I'm treating that as a second. I think it's the same difference. I just talked about... But then, yeah, sorry. I'm sorry to battle. But the connections to flow or current, I'm sure,

has to be there. But I personally don't have a solid sense of how that works. JOHN MUELLER some of the stuff that Deloes and Gattari were talking about in that you sort of, I haven't got a very precise example, unfortunately, but you sort of like define this plausible, recognizable field of things, and it's sort of bigger than just four attributes

interacting, like in a double bind. It's sort of like more oceanic or something. and then they're like, ah, but actually didn't you realize that there's these contradictions hidden in this field that you're oversimplifying things by just thinking of it in this basic mesh. But, yeah, I mean, that's... Yeah. It requires some more serious sort of digging. But, yeah, okay, that's useful. Thank you. Yeah, I mean, I've got to do a lot more digging, too, on this, but there's a whole rereading. I'm absolutely confident that certain capitalism and schizophrenia pieces,

I'm thinking, like, absolutely, especially the Georgian morals, are going to really spit out a hell of a lot. Because I think that piece in particular is really giving you some good sort of mechanical support for this. And obviously the whole structure of content expression and substance forms and substances is so explicitly diagonalizing once you start looking at it. But I'm still to do that re-waking process of math. It sort of went over my head.

Another thing is on UF blog as well, Urban Future, that you say the null hypothesis dramatized. all hypothesis dramatized. Has anyone seen this where it's basically like just a website where they have a message? You know, they're waiting for a message from a future time traveler to come back and then, you know, if we have like a, you know, this surely should be possible in the future at some time if singularity occurs, then we should be able to tell her. And there's nothing there yet. They're still waiting. They're still waiting with this sort of message bang. I was wondering what would Anthropole, if they could put a message there like thinking of putting a message

there for the past to see is it Anthropole is a Trojan horse? That's mine. But Yeah. That guy, the guy is from got this guy called Chris Pickover, and he's an absolute goldmine of all kinds of weird stuff. I mean, I really scavenge masses of stuff. And this weird thing, it's so utterly bizarrely non-humor. You just don't know how much humour is supposed to be in it. You know, how much of the kind of paradoxical structure of it is really supposed to be in your face

and what he's been communicating about that. Because obviously, like you say, still waiting for signs of time travelers from the future is just immediately you're in this kind of lurching spiral, aren't you? Of course, if it wasn't already subverted, it's never going to be subverted. So that just blinking silently to say, time-travel threat is not yet emerging. It's like, there's something I find hugely comical about that. If Anthropole could be there, I mean, I sort of,

that's also interesting, I kind of never really I've always sort of imagined maybe just having watched I always tend to see Anthropole as the victim of time anomaly rather than the perpetrator of time anomaly. But that's probably prejudice on my behalf. That's good enough. I'm really stuck on this practical way of thinking about the problem of time travel and relation to artificial intelligence, which I guess I could say that if a simulation believes that it exists a thousand years from now, because it has successfully predicted everything that will occur in the universe

up to a thousand years, then what's the difference whether it actually is in the future or not? There is no difference. Yeah. It potentially is in the future. So that's the way I think about it. But I actually just wanted to, I shouldn't have said that, because I really just wanted to ask you a question in relation to what you kind of last said about Deleuze, which is, and forgive me, I think I'm just slow, or I haven't done the readings I needed to with regards to both diagonalism and orthogonality, but with regards to Deleuze and foreign substance and all this stuff in the geology of morals, if I'm not mistaken, you just said that this is a kind of diagonal argumentation. But I take it, and correct me if I'm wrong, because I take it to be exactly the opposite, it, that Kant, yes, starts out with the diagonal kind of argumentation, as you said, in the

first class with his distinctions between the a priori and the posteriori, and the analytic and the synthetic, but that with regards to the geology of morals, D and G are, in a way, like deconstructing the diagonality there, that they're actually one and the same, like that a priori is a process... I can't actually... It's so anti-Kantian, I can't even... It's unthinkable. Isn't it like... Right? That, you know, like, every... the double articulation is both the first articulation is both the second articulation, and the second articulation is also the first articulation.

and isn't that anti-diagonal? No, no, that's diagonal! I'll tell you what, I think it's diagonal, which is really simple-minded, so I apologize if that's wrong. Which is just that, I think they're quite explicitly saying there is a natural or traditional alignment of form and substance with expression and comp you know that those those to I'm oppositions you know and they obviously this will refer to him slap and I think it's kinda one of these things that they often do its kind of almost explicit but not quite it's but you know him said isn't a susurian because

he has divided what in I'll she in classic just your instructions in the straight forward form content I'm you know what I call high the more model and they have that they separate it comes that separate into these to oppositions that then seen as I'm ways I someone and it plays I saw it's almost just out because of the genesis you know they but just because they been actually the terms are decided because they think all are on each other and then they are turned into a matrix and you know that and much because they don't say yes you've got

a all the substances of content and all these actions says all the expression so you you've got that now I'll see a two-dimensional matrix and the diagonal line of that matrix is the one cuts across what would have seen should be straightforwardly paradoxical self-contradictory hybrids so I'm so by filling out that table they they draw a diagonal line excluded by the previous articulation and then retrospectively I think the inability to see that are online is their critique of structural discussion.

You know, their critique of structural linguistics, their critique of Cisuer, is the fact that what they have now exposed as that diagram, which they attribute generously probably to Kemslev, is the critique. So that's where I'm seeing a type of diagram working in that piece. Cool, well thanks. Yeah, I'm going to rewind what you just said like 10 times. And then... Yeah, I mean if... Ask another question. To be honest, it's really simple-minded. I think if you just like... No, I totally agree. I'm trying to figure out how to map it onto like Cantor's diagonal argument.

And I'm like kind of having confusion as to why we're thinking of it in terms of a diagonal argument. But I will get there. Well, there's also – this is something I thought is – there's also a way of looking at what Kant is doing at the very, very beginning of the project in the first critique as an orthogonalization as opposed to diagonal. So like prior to Kant, there is an absolute diagonal – there's an absolute link between the operai or I in the analytic and the op-posterior I in the synthetic. And then he says he expands it into a matrix in which there's still a diagonal line below which there's a category that he thinks and everyone pretty much since almost has thought doesn't exist, which is the op-posterior I analytic. But he says the synthetic,

the synthetic operai or I exists. He prizes the terms apart into orthogonal axes that define a matrix. and it also, by opening up this category, defines a problem that he sets himself to solve and everybody sends, which is the construction of all of these synthetic a priori that structure experience and arguments for what they are and why they exist. And if we sort of compare that to what's being done by Bostrom in Superintelligence with the orthogonality thesis, he is prizing apart he's prizing apart these two things intelligence and goals into two infinite series and saying, I mean he doesn't actually say this part but implicitly saying that there's this diagonal line which is

and everything above it is intelligence succeeds goal and everything above that line is possible, everything below it isn't and so now there's this huge infinite triangular expanse of options, which include things like paper clippers and genies and oracles and skynets and all of this stuff. And in the same way that Kant says, well, now I'm allowed to talk about synthetic opriorize, therefore I'm obligated to because it's a real problem. Here's my crack at it. Bostrom says, you know, these AI monsters exist. These are real threats. Like we have this real problem, and that problem is this infinite triangular expanse of scenarios, which have now been opened up by the orthogonality I'm claiming.

So here's my crack at trying to approach these threats and that we're kind of now we're in the same... That's what we're in the wake of. Yeah. That's actually really helpful. That's really interesting. I would just sort of respond to that by saying I think that the diagonal component of that diagram that you've so loosely explained is the basic drives component. The fact that you say it exactly like this, that there is a kind of competence criterion for goals at this point, and so the two are connected and can't be fully settled.

And it's only after you leave that point that you're then free to explore this world of orthogonal monsters. So, yeah, this is a really interesting way of articulating that. Yeah. It's also interesting that, sorry, I'm just continuing the same train of thought because I can't help it, that if you think of what we're doing, right, When we're saying that this orthogonality that you're claiming is bullshit, here is a better diagonal, right? It's almost like redrawing a different diagonal line through this matrix. Every redrawing of a new diagonal is also a reordering of both axes, it seems to me,

or forces one that you can only have a... So if we're constantly recutting this grid to produce a new, you know, triangular growing expanse of possibilities that we want to deal with, then it can't be the same grid with the same two infinite axes of intelligence and, well, if we don't really want to talk about goals, just say behavior lines, telec vectors, something like that, that in order to cut, so you cut one for like the simplest that I said was that like you can't have, something whose goal is more, is unachievable by its intelligence level. So the simplest possible, biggest possible triangle. Then you've got the next one, which is basic

drives. And then if we go into like reflexivity, you know, and it's impossible to have this because you have to have a certain kind of reflexive loop, a certain way of regarding itself. It has to have a cephalized subject, or doesn't, or you know, if you're Peter Watts, whatever it is. Each of those is cutting this grid, but in order to do so, the things, for the space to still be above that line, the order of the two axes of intelligence, well, the order of the infinite collections that form the axes has to be changed. And it's interesting, there's something there, do you see what I'm saying, Nick? That every cutting of a new line is the recutting

of the orderings that you're using to define the matrix. I don't know, there's something very Duluth-y in there. Well, the ordering, I think I need to sort of walk through this a bit more slowly. Because, look, okay, the cognitive capability access seems to me, is that not rigidly ordered? I mean, however you're trying to measure this machinic intelligence, you have some kind of quantitative criteria that puts it, doesn't it put it in an inflexible order on that line?

I mean the difficulty I think it's a beautiful grid this thing because it does bring out this as well but it's very unbalanced in the sense that on one hand you seem to have a very neat arithmetical series of some kind quantitative series and on the other one it's very difficult even to construct it originally as a a clear ordering like an ordering there can be any particular intelligence can obviously have an ordering of its own goals and any utility schedule does that, but to have some sort of absolute ordering of goals, so in order to draw this cosmic

possibility space of the things that something might want and to put them in a linear series is a daunting I mean to me right now it's extremely daunting task. I'm trying to remember how Bostrom even does that, because of course you're totally right that he absolutely does. And I'm trying to think, what is the value, what's the metric that is actually organized in that second line? I don't know what, I don't remember what he says. I mean, my mind immediately jumps to, like, you know, you want to assume that it's temporal,

that it has something to do with, like, the schedule of your own, the schedule of, you know, your emergence, or you're achieving ever greater levels of capability, and so, but that's me just, like, naturally, you know, habitually diagonalizing again, is that if your goals are all about this self-improvement cycle, then, and maybe I'm just, this is me just drawing another diagonal line, I guess, and another. So the two kinds of axes we have, so intelligence is only like a strict quantitative schedule if we collapse the space of intelligence onto a line. So as he says, there are different superpowers. You can be intelligent at different things. Winning is a different... There's a different point scale of winning for every game.

or there is one time each time there is a point scale of winning for each time a game is played in the real world. So in order to say that intelligence is this generalized ability to be good at winning shit is a way of collapsing many lines into a single abstract line. So if you're saying that that whatever this line is, it is one which has a strict ordering which changing the diagonal we're drawing cannot change because that's what it is then that means that the other one, the opposite axis, would have to change what it is every time we cut a new diagonal line. And when he draws his, when Bostrom draws his, it's goals is the other one, and then which you can convert into a utility schedule if we cut it.

So goals, utility schedule, if we cut it in terms of this sort of self-improvement, reflexive foon scenario than it's steps of self-realization, steps in the becoming of some higher, more turbulent thing. I don't know. Honestly, I'm not going anywhere that's actually super relevant to the course right now. I'm just trying to... This geometry... I think this is really good, and I think it introduces lots of complicated things. I mean we haven't talked about this whole question about quantification and intelligence, which is obviously a problematic issue and it could be approached in lots of ways. So that for sure is a, you could say, an off-ramp, whatever, depending on whether to follow it.

And then this question of how to draw these matrices. I mean, it seems to me that it's easier on the FOOM side, way you just have a kid in the sense that you have much to axes in something my a complexity or sophistication much up structural organization is mom and I'm behavioral competence is gonna be behavioral companies as a kind of standing intelligence is gonna are you know what what you're mapping on the diagonal is that there is that your competence and self-improvement is being diagnosed against the structural section that your more you'll realize

and this is obviously what is and you know there said that the better constructed you are the more capable you are which means gathered improved constructions and and and that's that's the circuit so it seems to me well relatively straightforward but when you go into Boston territory it seems to me to get much more complicated because once you've got these totally arbitrary goals how do you do this linearization I mean the other thing that really comes to mind to me is that sort of

the way the diagonal arguments work with Cantor and with Turing is use the diagonal to demonstrate the existence of something that is not intuitive, that you didn't think existed before that proof, right? So like something uncomputable, which is a relatively unintuitive concept for people, actually, I find. And, you know, so Turing is constructing that diagonal to show the existence of one uncomputable thing, but actually the follow-up work now shows that there's a whole universe of uncomputable programs,

there's uncomputable numbers, and all sorts of weirdness that you can get out of that. So it's almost like, what you reveal from that is not itself or but but you sort of make a rip in this field that you you define and that that opens up this new conceptual space so it's only like I really like all takes sort of geometric framing what Bostrom is doing but maybe the result of what he is getting is not geometric the result is here's this weird new space that I've opened up and then mapping that as a follow-up. Yeah. But the two axes, when you draw the matrix, you have series that are at least considered unproblematic

at that point. I mean, I totally agree with you, obviously, that what's coming out of your diagrammization, what you're demonstrating is something in these cases especially, but probably generally, is something that is... It doesn't really exist in geometric space. Like you say, an uncomputable function, a number that is beyond countable infinity, the all of these products of diagonal argument do not themselves actually occupy the geometrical

terrain that you construct. I totally agree with you on that. But on the other hand, I think the series that are drawing that and that you're resting the whole thing on are absolutely unproblematically serial, well-ordered. I mean obviously it's easiest in Cantor's case where you simply basically have the number line that's just split apart in the sense of whether it's treated as sort of lexicographically or numerically which are completely interchangeable in arithmetic but you know you can peel it apart and so you've got a list of numbers and the actual value of the number and there's There's no question at all ever about which number belongs in which order in relation

to another number, even if it's sort of infinitely extendable to the point that you can't see, you can't comprehend that number, you still know the very fact that there's a number means it must be well-ordered. This Bostrom grid, the problematic axis of this Bostrom grid is it seems to me at least on the surface not at all like that. An axis that allows you to do a terrain of all possible goal-oriented behavior is like what is the well-ordered definition for that? If you think there is a goal sort of axis or a goal-related axis, then doesn't it come back to utilitarian definitions?

In that that's a very, at the heart of utilitarianism, you have this sort of functional definition of utility. and then there's all these arguments about what that exactly is, but the assumption there is a functional definition of utility that you can calculate and spit out some number. So isn't that very influential in this sort of idea that you can... Totally, totally, and of course it's nicely quantitative, and so, you know, utilities are a well-ordered series in this sense. But only in the abstract. De Gustavus non-disputandum applies. No, there's absolute utility. Isn't it kind of intrinsic that any absolute ordering of utility be diagonalizing?

I certainly take the first point that, you know, what's going to, by just having intensity of satisfaction or whatever is going to be the neat utilitarian quanta there is not going to give you any content in terms of goal orientation. That seems to me how that's not going to do the trick. But I mean, Jake's now the second point of this I think we might have to say more about. I mean, like in the sense that any ordering of goals, Any ordering of goals which is supposed to apply to any goal seeker,

so any intelligence in this context whatsoever, is going to be diagonalizing because the goals should not be ordered. There should be no ordering of goals with respect to anything else if total orthogonality is true, I guess would be. right not like not one by one yeah no that's worth hardening up into a into a proposition because deliver to the right person that would probably knocks out where yeah it is at exactly the right moment be the key to total cognitive trauma I Okay, I mean, I think we're hitting the point where we are a little bit over time today.

Yeah, yeah, sure. How do people feel? Are people ready to... I mean, you're all in different time zones, so I don't know. I'm at 2 o'clock in the morning here, so that's my time. Yeah, 4 a.m. here, so I'm getting a little more excited. Oh, wow, that's impressive. Yeah, I'm... Okay. I've gone to sleep yet, and it's 10 a.m. We're on to a kind of final food next week. Thanks everybody and have a good week. Thank you. Thank you very much.