0:00

There is a way to get John

0:02

Syracuse to leave his house. For those

0:04

of you who remember in November, leave

0:06

us November, I traveled up, I

0:09

made a multi-hundred mile train

0:11

trip to go see Marco and try the Vision

0:13

Pro prior to release at a lab. And I

0:15

said to John in advance, hey, I know you're

0:18

not going to the lab, but wouldn't it be

0:20

neat if you came down to New York and

0:22

visited with us. I think that would be a lot

0:24

of fun. And John told me to go outside and

0:27

play hide and go screw myself. Those were his words

0:29

exactly. In those words precisely. And

0:32

so I thought there was no possible way

0:34

to get John to leave his house, even

0:36

for his beloved friends of years and co-workers

0:38

for years and years and years. But

0:41

it turns out John, there is a way to get you out

0:43

of the house and I am happy to report the

0:45

ATP reunion for the first time since 2019 is

0:47

a go. John,

0:50

what's going on? We are

0:52

going to WWDC. All

0:54

of us? At Apple Park. All of us. All

0:56

13. Sorry, all three of

0:58

us. Indeed.

1:00

I am extremely extremely extremely excited.

1:03

I cannot overstate how excited I

1:05

am, especially to see the two of

1:07

you fellas, but also to be able to go to Apple

1:09

Park. The only place I've ever been on Apple Park is

1:11

a visitor center. You know, the public

1:13

visitor center that you don't need like any special privileges

1:15

to get to. I'm excited that

1:17

we are press for the purposes of this

1:19

event, which I'm really really excited about. I

1:22

swear I have never done a press

1:25

WWDC. I don't think so. And

1:27

I meant to look through my badges to see if I

1:29

could find a press badge. Maybe I have and it's

1:31

just been so darn long and I don't recall knowing

1:33

me. That's probably true. But one way or another, it's

1:35

happening again and I am so

1:38

excited. So I am glad that you are

1:40

willing to make the trek instead

1:42

of just a couple of convenient hours down

1:44

the interstate and or train from Boston to

1:46

New York. You're just going to cross the

1:48

country instead and I appreciate the effort. Yeah,

1:51

I mean I have done press WWDCs. We've

1:54

all attended many WWDCs, but I've

1:56

never been to one at Apple Park and neither has

1:58

Casey. So that's what I'm most excited for. is

2:00

to be officially allowed onto

2:03

Apple's campus. And I

2:05

think they don't let you take real cameras, which is kind of

2:07

a bummer, so I'll just take lots of pictures of my phone,

2:09

but whatever. I'm excited to do it. It'll

2:11

be a fun experience. Yeah, looking forward,

2:13

not looking forward to taking the plane flight, but you

2:15

know, you do what you have to. All

2:19

right, let's do a little follow-up. I

2:22

wanted to briefly call attention to there's

2:24

some new Vision Pro content. What,

2:27

really? Yes, there is. There's new Vision Pro

2:29

content. Who'd have thunk it? So

2:31

there's three things I wanted to call everyone's attention to,

2:34

two of which are from Apple, and one of which

2:36

is not. First of all, there

2:38

is a new sizzle reel. I

2:40

don't know if that's the bestest way of describing it, but

2:42

basically it's like a three and a half minute video that

2:45

gets you interested in the Vision Pro.

2:47

And previously, I hadn't watched it in a couple

2:50

of months at least, but previously my recollection is,

2:53

or the way I remember it was, it showed like a

2:55

little bit of the kids

2:58

playing soccer, rhinoceroses, it showed

3:00

the tightrope walker, and

3:02

I think it showed a very brief

3:04

bit of sports if memory serves. I

3:07

don't entirely remember, but one

3:09

way or another, there's a new one, and

3:11

I think it's really well done. It's still,

3:13

it's a little jumpy for my taste, just

3:15

a touch. It's not like that MLS thing

3:17

from a month or so back where it

3:19

was way too jumpy. It's just a touch

3:21

jumpy, but the soundtrack is excellent, and

3:24

I really, really like it. Again, three and a

3:26

half minutes. You can find it in the Apple

3:28

TV app, and since it's new,

3:30

it is kinda front and

3:32

center. I know I was complaining and

3:34

moaning about the information architecture last week,

3:36

but in this case, it's pretty good, so you should

3:38

check that out. Additionally, there's a

3:41

new, I believe it's the Adventure

3:43

series. This is the one

3:45

that had the tightrope walker on it.

3:47

There's a new episode of that all

3:50

about parkour, and it's three, I believe

3:52

Brits based on accents, although

3:54

who knows? Anyways, three

3:56

Brits doing all various

3:58

and sundry stunts. across paris and

4:01

it's really really well done it's like

4:03

twelve to fifteen minutes long someone that

4:06

much would and uh... and

4:08

i really enjoyed it it's not you know

4:10

earth shattering but it's really good and i

4:12

gotta tell you i know spoilers for the

4:14

twelve minute video but there's a point at

4:16

which the three gentlemen are trying to jump

4:18

from one rooftop to another and they position

4:21

the camera such that if you want you

4:23

can look down and see how tall it

4:25

is and i gotta tell you three

4:28

d and immersive it

4:30

looks area fell but what

4:33

pretty pretty cool and i think both of these

4:35

i mean that's it's all together these two things

4:37

are worth you know literally fifteen minutes a year

4:39

time i really do think it's pretty cool and

4:42

then finally uh... what if has finally launched

4:44

i had planned to do the whole darn

4:46

thing earlier today and report in on it

4:48

unfortunately they seem to release on pacific time

4:50

so it wasn't available until my afternoon and

4:53

i've only had the time to do the

4:55

first like handy ish minutes

4:58

but it was very very cool uh...

5:00

the premise here no spoilers is

5:02

you know you're in the marvel cinematic universe

5:04

and what if you know

5:07

things weren't the way they were in

5:09

the movies what if things went a

5:11

little awry and different and it's your

5:13

responsibility as part of the story to

5:15

try to fix all this and so

5:17

the portion that i've done so far

5:19

again first ten minutes or so is

5:21

they teach you how to cast some spells

5:24

lately is the interact uh... with

5:26

you so it begins as fully immersive you

5:28

can't see any of your own environment and then

5:30

it converts to uh... augmented such

5:32

that there are a couple of characters in your

5:34

space and i don't know if it was an

5:36

extremely happy accident or if it was deliberate but

5:38

i was doing this in my office and i

5:41

had spun my chair around for my desk and

5:43

monitors were behind me in my office i don't

5:45

know twelve feet by twelve feet from a concept

5:47

or something like four meters by four meters not

5:49

not terribly big not going to give me the

5:51

site and i don't matter not big at the

5:53

point and other public characters wanted designed to be

5:55

floating in the air and sure enough he was

5:57

floating in the air and his feet were kind of inside my guest

6:00

bed, which is behind my desk, but you

6:02

know, that's fine. It's he's magic. So whatever.

6:04

But then the other character was designed to

6:06

be on the ground and sure enough, he

6:08

was on the, you know, guest room floor.

6:11

Like he was standing on the floor. There's

6:13

a little shadow below him and it looked

6:15

just spot on. And then they had you

6:17

do some, you go back into an

6:19

immersive environment where it's, you know, all you can

6:21

see is what they're showing you. And

6:23

they have you do some spells where you're, you're

6:26

using hand tracking in order to do stuff. And

6:28

there were three or four that I learned and one

6:30

of them didn't really work that well, but the

6:32

rest were spot on. And again, I've only done

6:35

the first few minutes, but it's really slick and

6:37

it's free. So definitely, definitely check all three of

6:39

these out. Again, that's the new sizzle reel. I

6:41

think they just call it like immersive or like

6:44

immersive demo or something like that. Uh,

6:46

and then the parkour, the adventure

6:49

series, the second episode about parkour and what if by

6:51

Marvel and I've put a link to the, what if

6:53

thing in the show notes, assuming I can dig one

6:56

up. I should be able to, um, I

6:58

don't think I can link to the other ones in the show

7:00

notes, but if I find a way, I will do so. All

7:03

right. The low storage 13 inch

7:05

iPad pros have 12 gig Ram

7:07

or 12 gigs worth of Ram

7:09

chips in them, but they don't

7:12

use 12 gigs of Ram. John, what's going on

7:14

here? This is discovered by, uh, the,

7:17

I fix it folks with teardown, uh, their

7:20

lead teardown technician, uh, Sharon McCarthy, uh, found

7:22

that readers apparently, meaning like people who saw

7:24

the YouTube video, I don't know how they

7:26

spotted this, but they apparently spotted two, six

7:28

gigabyte Ram modules on the 256 gig and

7:31

512 gig 13 inch iPad pro. I looked at

7:33

the teardown video. I looked at the teardown on

7:35

the website. I'm like, how did they spot that

7:38

these were 12 gig modules? I can barely make

7:40

out the part numbers. They're so blurry from compression,

7:42

but whatever. Um, anyway,

7:44

uh, Sharon says our chip

7:46

ID confirms this with high certainty

7:48

to six gigabyte LPDDR 5X modules

7:50

produced by micron for a total

7:52

of 12 gigabytes of Ram. To

7:55

remind you, uh, Apple says this machine has eight gigs of

7:57

Ram because remember, if you want to get 16, you got

7:59

to get the higher. storage, they say

8:01

it has eight gigs, but there's apparently 12

8:04

gigs on the chips there. So why

8:06

does Apple utilize only eight gigs of RAM? I

8:09

think it says Apple has never done this before as far

8:11

as we know. Someone asked if it's possible these RAM chips

8:13

are defective and some of the RAM is disabled for some

8:15

reason. The folks at iFixit says

8:18

that, my understanding is that if this were the

8:20

case, they would receive a different part number and

8:22

be labeled as four gig, but I don't think

8:24

that's how LPDDR5 is manufactured anyway. There's very little

8:27

doubt that there's 12 gigs of RAM. Another

8:29

possible question, what if those are the

8:32

only LPDDR5X modules they can get their hands on, but

8:34

as soon as they're able to put in four gig

8:36

ones, they will. So just to avoid problems, this

8:38

initial batch is artificially limited. And

8:40

the iFixit answer to that was that these modules have been in

8:42

production since 2020 according to the spec sheets.

8:45

So it seems unlikely based on how long they've been

8:47

around. Super weird

8:49

because like,

8:51

why would they do that? Like, you know, if you

8:53

follow the thread, the Twitter thread about this, you can

8:56

see people throwing out a bunch of theories and then

8:58

basically getting shot down by iFixit. Why

9:01

would they use bigger RAM chips? Could they

9:03

not get smaller RAM chips? And

9:05

if they couldn't get smaller RAM chips, why wouldn't they

9:07

just free up the whole 12 gigs? The

9:10

other theories are like, oh, they're reserving a certain amount of

9:12

RAM for LLM stuff. Why would they

9:14

only do it on this model and not all the

9:17

models? Because I think this is the only one that

9:19

has more RAM and the other ones have the amount

9:21

that Apple says based on the teardowns. And

9:23

this one has this particular variant of this

9:25

particular model has 12 gigs instead of eight.

9:28

Super weird. But maybe this is

9:31

the side door for Apple finally putting more RAM

9:33

in its products. They'll just install it

9:35

and not enable it for you. But it's there.

9:37

You just can't use it. How is your M4

9:40

iPad Pro treating your eyes?

9:43

Because apparently it's not all roses and

9:45

pansies. I mean, it's good for me.

9:48

But Ben writes in to

9:50

say, I'm upgrading to an M4 iPad Pro from the 2018

9:52

iPad Pro almost immediately.

9:55

I noticed that my eyes seemed unable to

9:57

properly focus on the display resulting in eyes.

9:59

strain, fatigue, blurry vision, and even headaches. I

10:01

couldn't use the display for very long before

10:04

the symptoms reappeared, so I went down a

10:06

rabbit hole researching. This is kind of

10:08

like Marco with his in-pro thing, but even

10:10

worse. It seems like I'm not the only

10:12

one experiencing this, though. I have yet to

10:14

determine the exact issue. It might be PWM,

10:16

which stands for pulse-width modulation. And by the

10:18

way, when I follow these links to look

10:21

at the research that he was doing, whatever, everyone

10:23

just says, oh, it might be PWM. I think you

10:25

have a PWM. Yeah, it's probably PWM. People will come

10:27

into a forum or a Reddit or whatever and say,

10:29

hey, I just got a new M4 iPad Pro, and

10:31

the screen hurts my eyes. What do you think the

10:34

problem is? And people would say, yeah, it's probably PWM.

10:37

And I was like, are you going to explain what

10:39

PWM is? I mean, I guessed it was pulse-width modulation

10:41

just by knowing the term or whatever. But when you're

10:43

helping somebody, don't just say, yeah, you probably have PWM,

10:45

because they don't know what PWM is. Never mind that

10:47

the term pulse-width modulation doesn't make much sense. I bet

10:49

if you hear this now, you're like, well, I know

10:51

what pulse-width modulation is, but what does that have to

10:53

do with screens and why it would be hurting their

10:55

eyes? So we'll get to that in a second. So

10:57

anyway, Ben says it might be

10:59

PWM, though I've never known this to be a

11:01

problem. And I have been using LG OLED TV,

11:03

as well as OLED versions of the iPhone Pro

11:05

for years without any issue, or maybe tandem OLED

11:07

is misaligned. There is such a thing. I

11:10

ended up going to the Apple Store and compared my device

11:12

with others. Mine appeared to be slightly different, as if the

11:14

HDR was turned on all the time. Overall,

11:17

the display was always just too much, as best

11:19

described as basically yelling at me all the time.

11:21

Since I was still within the 14-day return period, they

11:24

switched my device to replacement, which seems to be much

11:26

better now at all and not perfect. My question is

11:28

for John, are you noticing any eye fatigue with the

11:30

new Pro, especially compared to the 2018 version? If so,

11:32

do you expect this could be improved through software updates?

11:35

So here's the research on

11:37

what PWM is talking about. According

11:39

to these links that Ben provided that we'll put

11:41

in the show notes, the

11:43

way this OLED and some

11:45

other OLEDs handles brightness,

11:49

there's two ways OLEDs can handle brightness.

11:51

One is they can send less voltage

11:53

to the pixels and they're not

11:55

as bright. So if you dim

11:57

the brightness of the screen or whatever, how does it do

11:59

that? do that. One way you can do it is

12:01

you can just send less voltage to the screen and it

12:03

gets dimmer, right? But if you

12:06

use, if you only use adjusting the electricity

12:08

going to the pixels to the control brightness,

12:10

apparently when you get to low brightness levels,

12:13

you lose a lot of the color saturation too and it

12:15

looks kind of like dingy and gross. So they

12:18

tend to not want to do that

12:20

for the lower brightnesses. The

12:22

other way you can control brightness on an OLED

12:25

is you can have the screen be at

12:27

maximum brightness briefly and then

12:29

go off and then max brightness and off and

12:31

the longer, you know,

12:33

the on period is, the brighter it

12:35

is. So if they

12:37

show like a little graph and this is the pulse width modulation

12:39

of saying, you know, if you just pulse the screen, pulse, pulse,

12:42

pulse, pulse, pulse, the faster the pulses go, the brighter the screen.

12:44

If you go pulse, pulse, pulse,

12:47

it is dimmer because the light is on less

12:49

of the time. You don't notice

12:51

this because it pull Apple screen this

12:53

time. Oh, it pulses at 480 times a second, which is a pretty

12:55

high refresh

12:57

rate if you remember from the CRT days like,

12:59

oh, my screen looks flickering at 60 Hertz. I

13:02

can see the flicker but 85 I can't see

13:04

it anymore. At 120 I definitely can't see it.

13:06

At 480 I can tell

13:08

you I cannot see OLED flickering.

13:11

But the complaint about the M4

13:14

iPad Pro tandem OLED is that it

13:16

uses pulsing to

13:19

control its brightness through its entire brightness range.

13:21

Apparently even at maximum brightness, it's still pulsing

13:23

as opposed to some other OLEDs which will

13:25

use pulsing down at low brightnesses. But once

13:28

they get the high brightnesses, they will do

13:30

that by keeping it out all the time

13:32

but it's sending less voltage. I

13:34

don't know if this is just the way tandem OLEDs work. Is

13:36

this the way Apple is choosing to make it work? But some

13:38

people report that this

13:41

bothers them. I don't personally see

13:43

how it could because I am not aware of

13:45

anyone who would notice flickering at 480 Hertz. We're

13:47

not even talking about motion here. We're

13:50

just saying like put on just a full

13:52

field, you know, red slide.

13:55

I can't see it flickering at 480 Hertz.

13:57

If you had asked me whether this is

13:59

flickering to control brightness, I would have said

14:01

no, because I can't see it. But maybe

14:03

people with very young or better eyes than

14:05

mine can see it. I

14:07

don't know. Or maybe it's anyway, follow

14:09

the links, decide for yourself. I,

14:12

and by the way, I would say, do iPhones

14:14

use this or do they use voltage regulation? I

14:17

honestly couldn't tell you because it just looks like

14:19

a screen to me and I can't see whether

14:21

it's dimming or pulsing at 480 hertz. Some

14:24

of the OLEDs that are out in the market

14:26

pulsed even higher rates than that. So yeah,

14:29

is this an issue? Is this something people are

14:31

imagining? Are some people just uniquely sensitive to it?

14:34

All I can tell you is that my old

14:36

man eyes don't see this and aren't

14:38

bothered by it. But we'll see. We'll

14:40

see if Apple has an update. There was, we

14:42

didn't put this in the notes for last episode,

14:44

but there is some kind of actual software error

14:47

with displaying HDR video where some of the highlights

14:49

are getting blown out that Apple, uncharacteristically,

14:51

Apple immediately acknowledged inside a software fix

14:53

was coming for. So

14:55

it is possible that maybe whatever issue people are complaining

14:57

about here will also be wrapped up in a fix.

14:59

We'll see. Michael Thompson

15:02

writes in with regard to trillions

15:04

of operations per second measurements or

15:06

TOPS measurements. Michael writes, I

15:08

found this article on the Qualcomm website that

15:10

suggests that the TOPS measurement they use for

15:12

their NPU performance is based on 8-bit integers.

15:14

In the paragraph headed precision, they state, quote,

15:17

current interest industry standard for measuring AI

15:19

interference, inference, excuse me, and TOPS is

15:22

an, is that int 8

15:24

precision. The context here being

15:26

whether or not the new surface line in

15:28

the, what is it, the copilot plus PC

15:31

or whatever it is, line of PCs,

15:33

are they or are they not

15:36

actually faster for neural related things

15:38

than Apple stuff? And

15:40

so I don't recall what was, does

15:42

this mean they are faster than I presume? Well,

15:45

I mean, so this, this matches what we

15:47

saw. The idea that, that on this website,

15:49

there's Qualcomm is saying the industry standard is

15:51

8-bit int 8 precision, right? And what we've

15:53

seen is Apple using int 8 and

15:56

so is Qualcomm and so is Microsoft and everybody

15:58

who's, who's current talking about the current. line of

16:00

products, they're all using 16-bit

16:18

stuff. And so their numbers were half as big.

16:20

But with their new stuff, Apple is using numbers

16:22

that are twice as big, and they're using Intate

16:24

and so is Qualcomm. So the answer is, everyone's

16:27

using Intate now. Is that because Intate

16:30

is more representative of the actual jobs

16:32

we're asking our NPUs to do? Maybe,

16:34

but whatever. The industry has

16:36

decided, when measuring tops, we're

16:38

going to use Intate precision. That

16:40

may become less relevant

16:42

if it turns out that the things we ask

16:45

our NPUs to do involve 16-bit or

16:47

32-bit values, and it doesn't really matter

16:49

how fast they can do stuff on Intate things, but

16:51

I would trust that Intate is actually a relevant measure

16:53

right now. So the answer is, the

16:56

Copile Plus species and the Snapdragon X

16:59

Elite thing has 40 tops. The

17:01

M4 has 38. Those

17:03

are both Intate measures. That means they're

17:05

essentially comparable. Eric

17:07

Jacobson writes in with regard

17:10

to iCloud Drive and Node

17:12

modules. So if you

17:14

recall, this was with John Sun, who

17:16

basically nuked his MacBook Air by trying

17:20

to sync the Node modules folder through iCloud

17:22

Drive. Eric writes, I haven't used it since

17:24

I don't use iCloud Drive, but there's a

17:26

project that will add a No Sync directive

17:28

to every Node module on a file system.

17:31

I imagine it might need to be rerun whenever a

17:33

new project is kicked off, and we'll put a link

17:35

in the show notes to No Sync-iCloud. Yeah, and

17:38

I tried to look at the code to remind myself

17:40

how. I think you make a directory with a .nosync

17:42

extension that has the same name as the other one.

17:44

That's the way you signal to iCloud Drive not to

17:46

sync the directory or something like that. This

17:48

is the Node module itself, so you can look at the code.

17:51

But unfortunately, the documentation is all in, what are we

17:53

going to say here? I'm going to say Chinese? Yeah, something

17:55

like that. Yeah, so the documentation is in Chinese, and I

17:57

can't read it. But the source code is not in Chinese.

18:00

in Chinese and I still couldn't quite make any sales of

18:02

it. But yeah, I

18:04

think it's just making .nosync directories in the right places.

18:06

And it's a node module that you can use, and

18:08

it will, I think, be just included in your project

18:10

and make sure everything knows sync. So that's useful

18:12

and helpful if you want to dare to walk

18:15

that tightrope of trying to use node modules with

18:17

iCloud Drive. Indeed. And then

18:19

Eric continues, I do, however, use Time Machine.

18:21

It can attest that the ASIMOV utility works

18:23

perfectly for excluding node modules and other dependency

18:25

directories. Also, it is a background service, so

18:27

it doesn't need to be reinitialized. And we

18:29

will, again, put a link to the show

18:31

notes, to ASIMOV. And

18:34

also, John, I guess you wanted to call attention to

18:36

the list of excluded directories, which I put in the

18:38

show notes as well. Yeah, it shows what

18:40

kind of things, like when it says dependencies, what

18:42

does that mean? Obviously, it means node modules, but

18:44

it has a whole list of all the different

18:47

things it excludes, things from Gradle, Bower, PyPy,

18:49

NPM, Parcel, Cargo, Maven, I

18:52

think CocoaPods might be, yeah, CocoaPods is in there.

18:55

Marco, have you heard of any of those things

18:57

other than CocoaPods? Precisely zero.

19:00

Flutter. Anyway, it's sad

19:02

that I think I've heard of all of

19:04

these. But I installed

19:06

this, and I ran it. It installs a little

19:09

launch daemon thing or whatever. And

19:11

it essentially does the, I forget

19:13

about the

19:15

time machine. I think it's an extended attribute or

19:17

something, but there's a way to exclude things from

19:19

time machine. Maybe it just calls TMUtil. Yeah. Anyway,

19:23

excluding all these directories from your time machine

19:25

backups can make your time machine backups go faster.

19:27

What it's basically saying is you

19:29

don't need to back up the dependencies

19:31

of your code. If you're writing something in node, and

19:33

you use 17 node modules, you don't need to back

19:35

those up. You get them from the internet anyway. You

19:38

got them through NPM or YARM or whatever. They're on

19:40

the internet. Do not back them up. It's not your

19:42

code. It's a dependency. You didn't write that code. It's

19:44

just pulling it in. And there are tons of files.

19:46

So if you can exclude those directories in time machine,

19:48

it will make your time machine backups go faster. But

19:51

who remembers, oh, what am I going to do? Go

19:53

to Options in Time Machine and drag the little thing

19:55

in there, set the extended attribute. I don't remember how

19:57

to do this. This just runs in the background. all

20:00

the time, looks for directories that fit this

20:02

signature, and excludes them from time machines. I

20:04

did that, I probably saved, I

20:06

don't know, thousands, many

20:08

thousands of files are no longer on my time machine

20:11

backups, because I ran this. I hope

20:13

it doesn't have bugs and isn't excluding a whole bunch

20:15

of important files from my time machine backups, but you

20:17

know, I've got multiple backup strategies. So

20:19

for now, I'm trying the experiment of running this

20:21

as I'm off-demon in the background to see if

20:23

it helps with my time machine backups. And I'm

20:25

still not running iCloud Drive, of course. And

20:29

then finally, this is actually, I should have

20:31

moved this up by the other Vision Pro

20:33

followup, didn't think about it, but anyway, Jonathan

20:35

Gobranson writes with regard to audio routing during

20:38

Vision Pro guest mode. So if you recall,

20:40

I was doing demonstrations for my mom and

20:42

dad, and I noticed that when mom was

20:45

on Mount Hood or whatever it's called, and

20:47

I had her go fully immersive, that the

20:49

crickets and whatnot were being routed through my

20:51

iPad Pro, which was doing mirroring at the

20:53

time. And so it doesn't really make for

20:55

a very good effect if the audio is

20:58

going through there. So Jonathan writes, you can

21:00

choose during setup of each guest user session,

21:02

whether to route audio to the Vision Pro

21:04

or the iPad or whatever the case may

21:06

be, if you choose to mirror content. And

21:08

we'll put a link to the Knowledge Base

21:11

article. So what you do is you look

21:13

up, and

21:15

you get the little green down

21:17

chevron near the top of your view.

21:20

Then you go into Control Center, and

21:22

you go back into the mirror my

21:24

view button. And then in there, there's

21:26

a audio routing section that

21:29

you can choose to push everything back onto the Vision Pro.

21:31

Not entirely sure why this isn't the default, to be

21:33

honest, because pretty much every time I've always wanted this,

21:36

but here we are. At least now I know that

21:38

there is a way around it. So good deal. We

21:42

are sponsored this episode by Fast Mail, not

21:44

only a great email host, but my chosen

21:46

email host since 2007. Obviously

21:49

long before they were sponsored, long before I was a

21:51

podcaster, I've been a Fast Mail customer, and I've been

21:53

very happy because hosting email, it's

21:57

constantly a moving target. You don't wanna do it yourself.

21:59

You don't wanna... to also use one of

22:01

the big services necessarily because then your name

22:03

is tied to them. I

22:06

like having my own control. I like being

22:08

able to have standardized hosting with a domain

22:10

that I own. Fastmail is a great place

22:12

to do this. They even now have domain

22:14

sales. You can add or buy domains right

22:16

through them and they set up all the

22:18

records for you and everything so it works

22:20

immediately and painlessly, but it's just a great

22:22

email host. Fastmail has all the features you

22:24

want under an email host. Of course, obviously,

22:27

it's iMap hosting and they have all the basic

22:29

default stuff that works with every mail client.

22:31

They also have all the modern productivity features

22:33

you might want, things like scheduled sends, snooze,

22:35

folders, labels, search, etc. They have all the

22:37

great plans, all the great features. I love

22:39

Fastmail. They now also have multi-user plans. If

22:41

you want to have your whole household have

22:43

a plan, they have a duo plan for

22:45

secure private email for you and a partner

22:47

at a reduced price and for just a

22:49

little bit more. They have a family plan

22:51

for up to six people. You can upgrade

22:53

your entire family. Of course,

22:55

in those family and duo plans, you have

22:58

things like shared domains if you want. You

23:00

can have shared or private calendars and address

23:02

books. You can have a family calendar all

23:04

shared, hosted by Fastmail, all based on standards.

23:06

It is wonderful. I strongly

23:09

recommend Fastmail and they are super private

23:11

too. For over 20 years, Fastmail has

23:13

been a leader in email privacy. They

23:15

believe in working for you as people

23:17

to be cared for, not data and

23:19

products to be exploited. Fastmail

23:21

has better span filters than everyone

23:23

else and absolutely no ads. It

23:25

is super easy. It gets to serve with

23:27

Fastmail today. You can download your data from

23:29

wherever you are now. You can import it into

23:32

your new Fastmail inbox. You don't need to leave

23:34

anything behind when you switch. It is great.

23:36

Try Fastmail free for 30 days and get

23:38

10% off your first year

23:40

at fastmail.com/ATP. Once again, free for

23:42

30 days, 10% off your

23:45

first year, fastmail.com/ATP. Thank you so much

23:47

to Fastmail for being an awesome email

23:49

host for all these years and for

23:51

sponsoring our show. you

24:00

too soon, but nevertheless, we

24:02

should talk about some last-second predictions,

24:05

and I guess this is most

24:07

predominantly from Mark Gurman in today's

24:09

episode. So I'm going to read

24:11

a whole bunch of stuff. One of you guys, feel free

24:13

to pipe in and interrupt at your convenience. But

24:15

here we go. Mark Gurman writes, Apple

24:18

is preparing to spend a good portion

24:20

of its worldwide developers conference laying out

24:22

its AI-related features at the heart of

24:24

the new strategy

24:26

is Project Grey Matter, a set of AI

24:29

tools that the company will integrate into

24:31

core apps like Safari, Photos, and Notes.

24:33

The push also includes operating system features

24:35

such as enhanced notifications. Much

24:38

of the processing for less computing-intensive AI features

24:40

will run entirely on the device, but

24:43

if a feature requires more horsepower, the work will be pushed

24:45

to the cloud. There are several new capabilities in

24:48

the works for this year, including ones that transcribe

24:50

voice memos, retouch photos with AI, and make searches

24:52

faster and more reliable on the spotlight feature. Faster

24:55

would be great, particularly on my iPad, please, and

24:57

thank you. They also will improve Safari web search

24:59

and automatically suggest replies to emails and text messages.

25:03

The Siri personal assistant will get an upgrade as well, with

25:06

more natural sounding interactions based on Apple's

25:08

own large language models. There's also a more

25:10

advanced Siri coming to Apple Watch for on-the-go tasks.

25:12

Developer tools, including Xcode, are getting AI enhancements

25:14

too. So let's stop here for a second

25:16

and look at this list of features, because we're

25:18

always like, how will Apple add AI sauce

25:20

to all this stuff? There

25:24

was a big story, German had it, and I think we might

25:26

have mentioned in the show, like, oh, they're going to fix Siri.

25:29

We were speculating months ago or weeks ago, whatever.

25:31

Is this the year that they're going to fix

25:33

Siri with AI, or are they just going to

25:35

add it to a bunch of other stuff? German's

25:37

rumor was like, no, they're doing a Siri thing,

25:39

so we can expect to see that. But here's

25:41

some specifics. Specifics seem not weird. I don't know.

25:45

Sometimes these rumors aren't comprehensive. Very often, Apple emphasizes

25:47

one or two particular things, whereas we just get

25:49

a laundry list and we don't know which one

25:51

they're really going to concentrate on, and it's going

25:53

to be impressive. But let's look at some of

25:55

these in turn and see how exciting they are.

25:58

Transcribing voice memos. Apple's been

26:00

doing transcriptions, for example, on voicemail

26:03

for a long time now. Having

26:05

transcription be better, that's

26:08

good. Probably not going to really radically

26:10

change people's licenses and existing features, and it's making a

26:12

little bit better. Retouch photos with

26:15

AI could mean a lot of things. That could be

26:17

a headlining feature. That

26:19

short description doesn't tell us. Is this going to just be

26:21

like, oh, they're better at background detection for tearing people off?

26:24

Or is it going to be like, oh, they're going

26:26

to really emphasize that they're doing the feature that Google

26:28

and many others have where if there's something in the

26:30

background or photo you don't want, you can remove it,

26:33

remove a person from the background who shouldn't be there,

26:35

remove a sign or a tree or something

26:37

like that. That

26:40

could be a big headlining feature. Google has certainly had

26:42

a whole ad campaign about theirs, and obviously it's

26:45

every graphics application from Photoshop to

26:47

Lightroom, Pixelmator and everything has

26:49

features like this, and they've been touting them, and I think they're

26:51

crowd pleasing. That's part of

26:53

the problem. So far, like, all right, voicemail

26:55

transcription and fancy AI photo cleanup and removal,

26:58

that's table stakes today. Neither

27:01

of those – I'm glad Apple's

27:03

getting there. They should, but neither

27:05

of those I think is likely to make

27:07

a big splash simply because other

27:10

people have been doing that for a decent amount

27:12

of time, and the rest of the industry is

27:15

there. What I want to

27:17

see from Apple

27:19

is what we get excited about from Apple. I

27:21

want them to blow us away with stuff that we

27:24

haven't seen from everyone else. I want to see features

27:26

that are not just playing catch up with the rest

27:28

of the industry. I want to see Apple leading the

27:30

way, not following, and these are just following so far.

27:33

Yeah, I mean, as with all these things,

27:35

like, we know that this has been done

27:37

elsewhere and even on Apple's own

27:39

platforms in various applications, but when people get

27:41

one of these new phones or they update

27:44

the OS or they see an ad

27:46

on TV that Apple touts this feature of erasing a

27:48

person, chances are good that

27:50

they've never seen that before if they're not a tech

27:52

nerd, right? And so as far as they're concerned, wow,

27:54

this is amazing. Especially if they don't

27:57

know it's even possible, like, they're just saying, oh, on my phone, I

27:59

could just tap a – person and remove them. And we all

28:01

know that that's been around for a long time and they're

28:03

playing catch-up. But A, they do have to play catch-up. We

28:05

don't want them to not do this. And B, it

28:07

can be just as impressive. It can be very impressive

28:09

to people who haven't seen it before. So I just

28:11

hope they do a good job of this because there

28:13

is a lot of competition, again, even on Apple's own

28:15

platforms in, you know, applications

28:18

that you can get for the Mac and for the

28:20

iPad and for the iPhone that already do the same

28:22

job. I hope Apple at least

28:25

matches them. Historically, Apple has kind of

28:27

been, they want

28:29

to make it simple. So there's not a lot of adjustability.

28:31

And if it does a bad job, like the Apple way

28:33

is like, well, just hop on the person and remove them.

28:35

And you'll tap on a person and it will make a

28:37

terrible mess of it. And you'll be like, is

28:40

there something I can do to try that again and do better?

28:42

And the Apple way is no. Nope. You tap

28:44

them and it does a good job. Good. And if it

28:46

doesn't, oh well. Whereas like in Lightroom,

28:48

there's a million knobs you can adjust and you can take

28:50

a second attempt and you can mask differently. You do all

28:52

sorts of things that are fancy your application. So there's

28:54

that. I just

28:57

really I really hope they do catch up and I hope

28:59

they do a good job of it. The

29:01

next time about improving Safari web search.

29:05

How? I get. All

29:08

right. Well, I mean, I can say I mean, for

29:10

whatever it's worth. Obviously, we don't

29:13

know what this means yet. We might not even learn

29:15

it in two weeks. But Safari

29:17

had like whatever Apple is doing,

29:19

whatever their back end logic is

29:21

for autocomplete suggestions

29:24

in Safari is

29:26

basically a search engine like they like I'm

29:28

I can't even imagine how many like millions

29:31

and millions of web searches Apple

29:33

avoids even making on the iPhone

29:35

by people just tapping that first

29:37

autocomplete review result that comes up.

29:40

So maybe it's just related

29:42

to whatever the back end is of that and

29:44

that could be one of those like invisible things

29:47

that everybody will take for granted and nobody will

29:49

even notice but we will appreciate it getting better.

29:51

You know, like things like autocorrect. Well,

29:53

notice if they screw it up like they've changed autocomplete many

29:56

many times over the years and when they screw it up

29:58

we all notice and we don't see it. I really

30:00

hope it doesn't start suggesting ridiculous

30:02

things. I think that might all be

30:04

local, too. So maybe that is a place for LLMs to

30:06

try to do better predictions. But anytime

30:08

they're messing with an existing feature, you hope it's

30:11

going to be a big improvement. But there's a

30:13

possibility, especially when you're throwing LLMs in the mix,

30:15

it's a possibility. The current strategy

30:17

is look at your past history, look

30:19

at the number one search results. They

30:22

could even be hitting the Google back end for some

30:24

of those things. It's a very straightforward, non-complicated algorithm. But

30:28

it's deterministic, right? And if they switch

30:30

from that to like, let's just chuck it over the wall to the LM

30:32

and see what it says. I'm concerned

30:34

that there might be some wackiness there, but

30:36

we'll see. More reliable

30:39

spotlight feature similarly. When you use spotlight on

30:41

the phone, I don't know if

30:43

it's using spotlight, spotlight. It's the same technology

30:45

as the run to the Mac. But it's

30:47

like, oh, I'm searching for stuff on my

30:49

phone. And it includes contacts and all your

30:51

applications and files and things you've

30:53

done recently. And yeah, you could

30:55

probably throw LLMs into the mix there to handle.

30:58

It's basically like, so you don't have to type

31:00

things exactly. And it's better about synonyms. And you can

31:02

type in vague things. Type

31:04

in some expression about what you want to do in settings

31:06

and have the LLMs figure out the setting you want to

31:08

go to. Potential for good, also

31:11

potential for messing up spotlight on the

31:13

phone. And then suggest

31:15

replies to emails and text messages. This starts to

31:17

get into the area where we thought, will Apple

31:19

go there? We haven't gotten

31:21

to chat bots yet. But obviously,

31:24

everyone else is doing this so it's another catch-up feature. But

31:27

the idea that Apple would, within the mail application or while

31:29

sending text messages, pop up a little clippy and say, it

31:31

looks like you're trying to reply to your friend. Do you

31:33

want me to write it for you? Which

31:35

is what, I mean, just look at the Gmail interface, for

31:37

example. Every time I'm in Gmail, it's like, do you want

31:40

me to summarize this email? Do you want me to just

31:42

write the reply for you? Like, Google's been pushing on this

31:44

for years. It used to just be like, here's

31:46

a canned phrase that is probably a reasonable reply to

31:48

this email. And now it's like, you know what? I

31:51

can just write the whole email for you. Like, you

31:53

don't have to click from canned phrases. You don't have

31:55

to, you know, the autocomplete that happens in like Google

31:57

Docs and everything. It's just like, let me just write the whole

31:59

thing for you. And that, we

32:02

should find a link to Nevin Mergens blog post. Nevin

32:04

Mergens said he got an email from a friend that

32:06

was written by an AI and he kind of

32:09

flipped out about it, right? That seems

32:11

like a less, that

32:14

is not a conservative move to suggest to

32:17

people that the phone will write their email

32:19

or their text message for it. And summarizing I can see

32:22

it's like the phone is helping me if I just want

32:24

to summary it, I don't want to read it all, let

32:26

the phone summarize it. It's like asking the phone to help

32:28

you deal with your stuff. But

32:30

having generative AI write something

32:32

that you will send as if you wrote

32:35

it yourself is a step

32:38

farther than Apple has gone. So we'll see how they

32:40

present that. Again, obviously everyone else is doing this, it's

32:42

not new. I'm not saying Apple is going to be

32:44

the first one to do this and they're going to

32:46

get in big trouble or something. It just doesn't, it's

32:49

not a very safe thing to do. So I'm wondering

32:51

how daring Apple will be in like

32:53

a little presentation. They'll just be like, and it

32:55

can even suggest a reply for you and some

32:57

happy person who's happy to get a text will

33:00

tap a thing on their phone screen and they'll

33:02

be so delighted to see the three word reply

33:04

come and they'll hit the send button and they

33:06

won't talk about it anymore. But

33:09

I don't know, it wigs me out a little bit too. I

33:12

mean keep in mind like we're probably not

33:14

that far away from, I mean people are probably

33:16

doing it now. Where like you're

33:19

using your AI to compose an email

33:21

that will be then read by the

33:23

recipient's AI and we will all just

33:25

be sending even more garbage crap to

33:27

each other and having it be processed

33:29

by even more garbage AI junk

33:32

on the other end and we will finally reveal email

33:34

to be the useless thing that has been most of

33:36

this time. It's not useless and I feel like

33:38

in a business setting this is most useful because there's

33:40

a lot of boilerplate and ICDs in business that would

33:42

help. But like we already have auto, like I mentioned

33:45

auto complete, like Gmail forever has had a thing

33:47

where you start writing something and it guesses what

33:49

you're probably going to say for these next three words and

33:51

you can auto complete that. But

33:54

I feel like that kind of piecemeal auto complete,

33:56

even if it is L-empowered, that piecemeal auto complete

33:58

at least forces you to read it. everything,

34:00

whereas having a button that says, compose

34:03

a reply for me, invites the possibility that you will

34:05

not read what it generated, because that will take too

34:07

much of your time, and you'll just hit send. And

34:09

now we're sending messages that we're not even looking at

34:11

to give a thumbs up or a thumbs down on,

34:14

because it just takes too much time. And that's just,

34:16

I think it's a waste of people's time, because

34:18

if you don't read what it wrote, maybe it didn't write what you

34:20

wanted it to write, and now you have an even more confusing email

34:22

thread. And I mean,

34:24

again, we'll figure this out as a culture,

34:26

as a technology advances, but I'm just saying

34:28

character for Apple, for Apple's thus far very

34:31

conservative approach to AI, suggesting

34:34

replies to emails and text messages seems like

34:37

a big move for this particular

34:39

company. Yeah, agreed. Continuing

34:43

from Mark Gurman, one standout feature will

34:46

bring generative AI to emoji. The company is

34:48

developing software that can create custom emoji on

34:50

the fly, based on what users are texting.

34:52

That means you'll suddenly have an all new

34:54

emoji for any occasion, beyond the

34:56

catalog of options that Apple currently offers on the

34:58

iPhone and other devices. Another

35:01

fun improvement on related AI will be the revamped iPhone

35:03

home screen that will let users change the color of

35:05

app icons and put them wherever they want. For instance,

35:07

you can make all your social icons blue or finance

35:09

related ones green, and they won't need

35:12

to be placed in the standard grid that has existed since day

35:14

one in 2007. All right, so that's

35:16

a lot of stuff here. The generative AI

35:18

emojis, obviously they're not emojis, because emojis are

35:20

like Unicode things. There's a defined set of

35:22

them. You can't just make up new ones. I mean, you can't. If

35:25

you're the Unicode Consortium, but Apple can't. So what it's

35:27

actually saying is kind of like on Slack and Discord

35:30

and all these other things, where you can

35:32

sort of generate custom stickers, or I think they call

35:34

them custom emojis or custom reactions or whatever. The point

35:36

is, this is going to be a new graphic that

35:38

they'll generate on the fly for you that will be

35:40

sent as an image, because they can't just send it

35:42

as a Unicode code point, because they're

35:45

not dynamically adding things to Unicode. That's not

35:47

how this works. So they're kind of generating stickers

35:49

for you. And I guess they're

35:51

going to like, you know, they

35:53

have a bunch of like Mr. Potato Head style building parts,

35:55

they feed into the LLM, and then it

35:57

can do like sentiment analysis to figure out what kind

35:59

of of emoji we'd want to say like there's

36:02

a lot of emoji especially for things like face

36:04

reactions and stuff like that I

36:06

guess there's ones that aren't there like you ever try

36:08

to someone someone emoji and you're shocked that there's not

36:10

like a watermelon emoji or something I can't think of

36:12

one that doesn't exist sorry but this frequently I go

36:14

into emoji and I'm like oh surely there's an emoji

36:17

for this and like there's no ice cream cone like

36:19

again I don't know if that's real or not like

36:21

I'm shocked by what's not there so that's nice that

36:23

it will generate to it but it'll have to send

36:25

it as an image because it can't send it in

36:27

the other way and I do wonder

36:29

this is another place where like I feel

36:31

like Apple is taking a risk even with

36:33

a very very limited image generator that's trained

36:35

on their own source you know trained on all

36:37

their own sources for emojis that Apple makes

36:39

and can generate new ones

36:42

based on that and some other information there's

36:44

the potential to generate a little image that

36:46

maybe is embarrassing to Apple as a company

36:48

you know I'm saying so and there's

36:51

a potential to generate images that

36:53

just look ugly or not up

36:55

to Apple standards you know what I mean like this

36:58

is that's a weird thing to be touting

37:00

and the coloring icons that's you

37:02

know another I'm not saying keeps it from wanting to say a bridge

37:05

too far it's not a bridge too far but it is a bridge

37:08

like you're gonna take developers

37:10

icons and draw over them they've

37:12

done this before they used to add the shine

37:14

to your icon if you didn't opt out you

37:16

remember that like the gloss yeah yeah oh yeah

37:18

like we're gonna change the color of your icon

37:20

like I'm sure these companies that are like so

37:23

proud of their branding and change their icons or

37:25

whatever like that now the user will be

37:27

able to say you know what I want Instagram to be green and

37:30

the OS will be like sure I'll take that Instagram icon and

37:32

I'll make it green we can do that that's

37:35

weird I mean you can make little color-coded things on

37:37

your home screen I suppose but yeah

37:39

I can't imagine that's the whole story to that one

37:41

I Mean First of all, like a

37:43

lot of companies I think that have a problem with

37:45

just like trademark issues with like what if you if

37:48

you color my icon green First of all, it's not

37:50

my brand identity. Second of all, it might look like

37:52

my competitors or it might infringe someone else's trademark or

37:54

something like I could see. there's I could see like

37:56

big companies having a big problem with that but well

37:59

if users are new. Doing in a matter whether

38:01

the pressing for this a Mac Os back

38:03

in a day, someone else will look up

38:05

on version of Mac Os I'm cursed or

38:07

have been the operating system for max. I

38:09

do this when you added a label to

38:11

something akin to label the Mccoist entering classic

38:13

back on sugar levels in one version of

38:15

Mac O S or System Six or whatever

38:17

was called back in it which of something

38:19

of sigma to be Mrs and seven when

38:21

you apply to label a collie the icon.

38:24

By. To review fried green green label I would

38:27

like green tint the I can buy basically

38:29

making the ball pixels greener, the bike pixels

38:31

rad. I think it did not look good.

38:33

It was not a good. learned that that

38:35

didn't last very long time. Certainly Mac Os

38:37

Ten era. that has never been the case.

38:39

but. After the user

38:41

deciding to color I can like and that

38:43

was not on my list of things to

38:45

think of they were be doing our noses

38:47

like unrelated day. I've alleges I was eighteen

38:49

change the next week we'll probably have many

38:51

more produces area receive features but what an

38:53

odd what an odd thing to do but

38:55

us has is this stuff people clamoring for

38:57

this. I want to change the color of

38:59

my icons. I mean since I can't they

39:01

added that while ago where you can pick

39:03

from different like on the Bob Lutz but

39:05

even that that member how conservatively the added

39:07

that like it has to pop have a

39:09

dialogue and people spaces. Are you know what's

39:12

happening so apps can masquerade as the rap?

39:14

I literally just implemented that underpin the rewrite

39:16

a certificate. Not,

39:19

but I'm but I mean the I'm that

39:21

There's clearly there is a very large market.

39:23

there was I going to before I? there's

39:25

a very large my god. Therefore, these homes

39:27

ring customization apps that use all sorts of

39:29

tricks and hacks and frankly kind of bad

39:32

hacks to get people to be able to

39:34

customize icons for their apps and make like

39:36

that their aesthetic home screens that they want

39:38

that A huge demand, Huge market. I think

39:40

Apple. Wants to try to address some of the

39:42

man without having all these you tax. If this

39:45

is indeed what they're doing here, the problem is

39:47

unless you can have. Arbitrary.

39:49

Icons for apps like to cook like specify your

39:51

own image as the user as he came with

39:53

circuits right like if you it if you can

39:55

do that then all these apps a solid. These

39:58

could see if they saw it seems. The

40:00

people are just drawing it themselves like their

40:02

use an app that have liked cool theme.

40:04

Packs of here's here's like a boat, a

40:06

pipe layer, icons for all the popular apps

40:08

that you know you have like Instagram plus.

40:10

Also here's a bunch of like random ones

40:12

you can apply yourself to whatever else you

40:14

need them to be in. have like ten

40:16

generic ones. That's. A huge

40:18

market and so if we're just talking about like.

40:21

You. Can tinted seven different colors. that's not going

40:23

to get rid of us market at all and

40:25

you know some people use about that are be

40:28

great but I can imagine that being worth the

40:30

hassle in the troubled that they might get him

40:32

with other companies so. I

40:34

have a feeling this is like or embryos. Sometimes a

40:36

rumor comes out. And. Birds College.

40:38

Ah, and then you know, whenever it's actually

40:41

announced we see, oh, there's important detail x,

40:43

Y and Z. Hear that explain it better

40:45

and that now makes more sense. I think

40:47

we're missing those details right now on this

40:49

feature. Add to their sub the something else

40:51

here to explain this that we're not getting

40:53

it. I'm in a might just be like

40:56

you mention those icon packs are part of limitations of

40:58

customize your on scream using shock of through place that

41:00

by com for thought zipper cumbersome as the doing anything

41:02

else can we get behind the existing app and make

41:04

the shortcut to be out in assists to the half

41:06

on second if you get one of those icon packs

41:08

maybe it of the have an icon for the app

41:10

that you want to change. But. Of

41:12

the O S says okay, well you're making a little

41:15

region of green icons and you can use items from

41:17

the steam pack you got. Oh but there's no green

41:19

icon for this app. will just take the exit. The

41:21

actual icon for the happened to degree outfits and. So.

41:24

Maybe it's like us, you know I catch all

41:26

for if there's if busy bargain draw their own

41:28

custom icons. and if we can't

41:30

find one in a prebuilt icon pack now you

41:33

can just the least make the bygones mats i'm

41:35

i don't quite understand the idea that to conquer

41:37

everything but there's that and have the other part

41:39

of this is being able to the says not

41:41

but i'm the standard good reasons we just have

41:44

to be able to leave specks you know to

41:46

make space or of clear space or icons anymore

41:48

i'm assuming that's what they're talking about here is

41:50

like now you can arrange things on the home

41:53

screen and leave gaps if you want to i

41:55

still think doing anything on the home screen on

41:57

the i phone as credibly painful talk to listen

42:00

for years and years, I think it will continue to be

42:02

painful. I really wish there was a nice

42:04

interface where you could mess around with this

42:06

stuff and do it without destroying anything and

42:08

either commit or roll back all your changes

42:10

as opposed to the current system, which is

42:12

just pure chaos. And it's like the hardest

42:15

game on the entire operating system to try

42:17

to rearrange icons on your phone without screwing

42:19

everything up and just throwing your hands up

42:21

and saying forget it. I'm hiding

42:23

everything into the app library. Yeah,

42:25

I wonder. I feel like I

42:28

heard this from Jason maybe. I

42:30

heard it somewhere. Maybe it was Mike on upgrade. I forget

42:32

where I heard it, but somebody pointed out, well, maybe there

42:34

will be layers and that

42:37

makes me think of how it works. That was

42:39

upgrade, yeah. In SF symbols, you

42:41

can have different layers and you can color them in

42:43

different ways and whatnot. I think

42:46

that makes some amounts of sense, but

42:48

that's not going to fix the Instagrams of the world.

42:50

So I'm not sure. Or maybe it will be opt-in for

42:53

developers, but again, I don't see it. Oh, Vision OS has

42:55

this, right? Don't they have layers in their icons? Yes.

42:58

They actually require it with Vision OS. Yeah, but

43:00

if they made that a new, like, hey, for

43:02

iOS 18, if you provide a layered

43:05

icon, our new

43:07

coloration API will respect that, like in case you're

43:09

something like SF symbols where we can colorize parts

43:11

of it. We'll see. It

43:13

just seems to me like a weird place to be

43:15

spending resources. I think the arrangement part is a smart

43:18

feature. They should have that. People have wanted that for

43:20

a long time. People have been doing it with spacer

43:22

icons. It's annoying. The place where they

43:24

really should have spent resources is how hard it is to

43:26

rearrange the home screen. We'll see if

43:28

they do that. Yeah, that's the only way I

43:30

really wish I could still interact with my phone

43:32

via iTunes, which I know is now Finder, but

43:35

that was so much better, was doing home screen

43:37

rearrangement. And even that was terrible, but it was

43:39

better. Yeah, it was terrible, but it was way

43:41

better. All right, finishing up with

43:43

Mark German, a big part of the effort

43:45

is creating smart rehabs. The technology will be

43:48

able to provide users summaries of their missed

43:50

notifications and individual text messages, as

43:52

well as web pages, news articles, documents, notes, and

43:54

other forms of media. There's also no

43:56

Apple Design chatbot, at least not yet. That

43:58

means the company won't be... competing in the

44:00

highest profile area of AI. And the version

44:03

that Apple has been developing itself is simply

44:05

not up to snuff. The company's

44:07

held talks with both Google and open AI about

44:09

integrating their chat bots into iOS 18. Apple

44:12

ultimately sealed the deal with open AI and

44:14

their partnership will be a component of the

44:16

WWDC announcement. So a couple of quick thoughts

44:18

here. First of all, I think

44:20

we all are in a group chat that oftentimes

44:23

will just pop off at a time that you're not

44:25

paying attention. And so it is not uncommon, particularly I

44:27

find that I'm in a group chat with a couple

44:29

of guys that are constantly

44:32

looking at cars that they may or may not

44:34

ever buy. One of them in

44:36

particular is obsessed with finding a affordable 9-11

44:38

and those are mutually

44:41

exclusive terms. You cannot find a 9-11 that is

44:43

affordable. That's not a pile of garbage. And

44:45

so anyways, so I will often come back to my

44:47

phone or computer, what have you, with literally 40 or

44:50

50 messages, most of which I don't particularly care about

44:52

because 9-11 is not really my cup of tea. And

44:54

so if there was a way to summarize

44:56

that, I'm all in, sign me up. And

44:59

then additionally, I mean, here it is that Gurman

45:01

is saying open AI is the winner. And like we discussed,

45:04

I think this past week, that's

45:06

going to be a little dicey. So I wonder how

45:08

they spin this. Well, in typical Gurman fashion, there's

45:11

also no Apple design chat bottle. It's not yet Apple

45:13

designed. Do they mean Apple's in-house LLM that they've been

45:15

working on? Is that what he's saying? There won't be

45:17

any of it because I bet whatever, like they say

45:20

they did a deal with open AI, but he doesn't

45:22

say, okay, well, what does that deal, how is that

45:24

deal going to manifest? Is it going to manifest by

45:26

iOS 18 having a chat bot that is powered by

45:28

open AI? And will that not be

45:31

Apple designed or will it be open AI designed?

45:33

Will it be open AI branded? These are all,

45:35

I guess questions that he doesn't have the answers

45:37

to yet. But yeah, the news is they did

45:39

a deal with open AI. There's also, you

45:41

know, news this week that like Apple is still talking

45:43

to Google, obviously, you know, anyway,

45:46

so we'll see what, what,

45:48

uh, if the words open AI appear at WWC,

45:50

if they're going to announce this as a partnership,

45:52

they're just going to use them on the backend.

45:54

Like I said last week, maybe they've been building

45:57

features using the existing open AI API. And now

45:59

they just think deals and how they can

46:01

officially do it in a released OS instead of

46:03

just doing in dev builds but I'm just not

46:05

quite sure how this is going to manifest because

46:07

this paragraph begins making it seem like oh there's

46:09

gonna be no chatbot but they have licensed the

46:11

open AI chatbot. Well is there gonna be

46:13

a chatbot or not? Is there gonna be part of

46:15

iOS where you can start typing to

46:17

an LLM and getting answers in the chat

46:20

GPT style or will there be an open

46:22

AI app or will it be integrated into

46:24

Siri? Questions we do not have

46:26

answers for yet but that is the the capper

46:28

on all this stuff. A bunch of strange features

46:30

which I mean they all seem plausible to me

46:32

but this is not the list I would have come

46:35

up with about how to add AI sauce to iOS

46:37

18 and at the very bottom is you know

46:40

open AI something something something

46:42

open AI. Yep so

46:45

yeah we'll see what happens as I think John said a

46:47

moment ago typically you know in the couple of days leading

46:49

up to WDDC we oftentimes or

46:51

maybe the week before we hear some

46:53

last-minute leaks and so on I was

46:55

gonna say announcements but they're not announcements so we'll

46:57

see what happens but I am very

47:00

much looking forward to sitting next to you

47:02

fine gentleman and drinking this all in while

47:04

at Apple Park hopefully not getting a sunburn.

47:06

Well there was did you see there was

47:09

one kind of last-minute thing that came out

47:11

today also about Siri being used to script

47:13

apps but not yet and only

47:15

Apple apps? Yeah

47:18

it was basically the gist of this rumor I think it

47:20

was also Gurman was just in

47:22

a separate report that Apple

47:24

will also preview Siri

47:26

being able to like take more complex actions

47:29

within apps kind of what we've been talking

47:31

about you know we've been talking we've been

47:33

speculating about how wouldn't it be great if

47:36

you know apps could expose you know

47:39

kind of stock actions similar to the intent system

47:41

and you kind of describe what they are in

47:43

some kind of language and then have an LLM

47:45

based you know Siri interface be able to analyze

47:47

what the app is telling the system here is

47:50

how you can do it and here's the actions

47:52

to call to do it and

47:54

be able to offer to users the ability

47:56

to perform actions like that via voice commands

47:59

that are not necessarily phrasing exactly the right way,

48:01

that we're not set up in advance as shortcuts and

48:03

things like that. The rumor

48:05

that came out today from German is basically exactly

48:08

that is happening, but

48:10

not yet. That basically that

48:12

it is, that exactly that kind

48:14

of thing will be previewed. It won't

48:16

actually ship until a point release, probably

48:18

next, they said next year, so 2025, so

48:21

presumably in the first half of next year, you know, iOS 18.2 or 3 or

48:23

4 or 5 or whatever, and

48:27

that it would only work in Apple's apps to start. So

48:30

that tells me, not an API

48:32

yet. Or an API

48:34

that is released immediately, you know, released

48:36

at WWDC, but obviously nobody's going to

48:39

be able to implement it until

48:41

at the earliest of the fall, and it may not

48:43

be an instant adoption.

48:46

That's interesting. You know, I wonder if

48:49

maybe we will see, you know, the current

48:51

API to do all this is called AppIntents.

48:54

This is, they launched just a couple

48:56

of years ago, and it's like the

48:58

new Swift-based method of Intents, which I've

49:00

mentioned before, is way better

49:02

for developers and way simpler and way

49:04

easier to implement than the old system

49:06

of these weird Xcode files that had

49:09

custom configuration that would generate files, and

49:11

the generation process would always break and

49:13

cause weird bugs, and then you'd have

49:15

this extension that processed it and would

49:17

have to communicate to your app via

49:19

weird signaling mechanisms for extensions to apps. That

49:21

was a whole mess before, and they've redone

49:24

that system two or three times. I

49:26

wonder if, you know, sometimes when Apple is

49:29

kind of going in a direction before a

49:31

future update for something, sometimes

49:33

the APIs will launch for

49:36

it that don't come out and

49:38

explicitly say this is what this is for, but

49:40

kind of lay the groundwork for it. So

49:43

for instance, AutoLayout coming out, this is the

49:45

famous example, AutoLayout coming out before the iPhone

49:47

6 came out with its first, like, really

49:49

new screen size. You know, not counting the

49:51

iPhone 5, of course. That was just adding

49:53

a table row. Anyway,

49:55

so that's – I wonder if what

49:57

we're going to see this summer is maybe – maybe

50:00

they will add something to the AppIntense

50:02

API to give us more

50:05

ways to describe to the

50:07

system what these actions

50:09

do and maybe hide them from the

50:11

user but make them available

50:13

to this indexing system that presumably the new

50:15

Siri would have. Maybe we'll

50:18

see something like that. So I think

50:20

this, if for some reason they

50:22

demo this and don't tell us how to use

50:24

an API for our apps in the future, maybe

50:27

we'll be able to see hints in like

50:29

what's new in AppIntense this year. So

50:32

keep an eye on that. It sounds

50:34

more like they're considering their

50:36

own apps first. They're not quite sure what that API

50:38

should look like so they can experiment with their own

50:40

apps because if they change their minds, they'll just rewrite

50:43

their own apps to fit with it or whatever and

50:45

once they get it worked out, because it's very often,

50:47

especially in the old days of Mac OS X, they

50:49

would very often roll out entire new frameworks. There would

50:51

be private frameworks that only Apple's apps could use and

50:54

they would work out the kinks. Is this the right

50:56

API? Because they could change the API all they want because

50:59

no one else should be using this private framework and

51:01

once they finally got the framework where

51:03

it looking like they wanted it to,

51:06

then the next year they roll out essentially the same framework

51:08

under a new name but now it's a public framework. It

51:10

kind of sounds like what they're doing here, that they're not

51:12

at the point, like especially if they're kind of like rushed

51:14

to catch up, they're not at the point where they're ready

51:16

to even perhaps even to tell developers

51:19

in the size class type of way, do this thing and

51:22

then next year you'll be folded into the system because it

51:24

seems like they don't even know what that would be yet.

51:26

Like how to indicate, like they're working that out themselves but

51:28

they do want to get something out there and you can

51:30

demo it if they do it on their own apps. They

51:33

say coming in spring or whatever, not

51:35

an 18.0, whatever released down the road, we're

51:39

gonna have this and it's only gonna be our apps to

51:41

start basically because they haven't even figured out what they want

51:43

to ask developers to do and

51:45

related to that, maybe think of Windows

51:47

and the recall feature or whatever they

51:49

have. Isn't there something on iOS that's like activity,

51:52

something rather where you essentially make

51:54

a call let the OS know

51:56

what your app is doing. Yeah,

51:58

the user activity. framework.

52:01

Yeah, anyway, there's a similar, we had a very similar name

52:03

on Windows. It's some kind of like activity something something. I

52:05

think I have the exact same name. I

52:07

saw the story of reason behind like user

52:09

activity. That's our API. Yeah, and it's like,

52:11

it's basically that's how they did the thing

52:13

where like you can see like a PowerPoint

52:15

slide and recall and click on it and

52:18

go to that slide in PowerPoint. It's because

52:20

like during the, during the

52:22

screen recording like PowerPoint itself called the

52:24

user activity API to let it know

52:26

that it's viewing slide number 123 and

52:28

like recall recorded that because it's an

52:30

OS API and you're just telling the OS what's going

52:32

on and then when you click on the thing and

52:34

the recall thing, it looks back at that timestamp and

52:36

see during that time PowerPoint had this activity user activity

52:39

thing and it jumps back like same type of deal.

52:41

There's cooperation between APIs of

52:43

the app has to call that just says here's what's going on. Here's

52:47

my deal. And then another part of the system, you

52:49

know, cooperates with that sees the

52:52

stuff that those APIs called and connects the dots.

52:54

And that's exactly how we would think. I mean,

52:56

the thing you described for user intents, I

52:58

think that's probably better than when Apple is going to roll

53:00

out. I have dim hopes that they're going to do what

53:02

you describe what you described. I think is what they should

53:04

do, but it sounds much harder than what they

53:06

might do this, especially since they're

53:09

not even rolling out publicly. It seems like it might be a little

53:11

bit simpler than that and like not

53:13

involved the existing intent system at all,

53:15

which would kind of be a shame. But you know,

53:17

we'll see like that when they demo the feature, we'll

53:19

see the utility of it because they won't tell you

53:22

how it's implemented. But I do hope there is a

53:24

developer story for this. There's not, you know, people are

53:26

going to have a lot of questions of like, hey,

53:28

you demo this feature in the keynote, but it's only

53:30

for Apple things. Is that coming for third party soon

53:32

and they're going to not comment on

53:34

future products? Yeah, I

53:36

hope this is a direction

53:38

they're going. I was very I was

53:40

simultaneously very happy to see this rumor come today,

53:43

but also a little bit disappointed that we probably

53:45

won't be able to really use it until next

53:47

year or something like next year, because

53:50

this is an example of something that only

53:53

the platform owners can do and

53:56

it takes advantage of Apple strengths and

53:59

you know and kind of doesn't really rely on their weaknesses so

54:01

much, and gives them a

54:03

way to lead again. I

54:06

think so many of Apple's AI features are

54:09

going to be rightfully seen as playing catch-up,

54:11

as I was saying earlier. They

54:14

need to show us that they're not, especially

54:16

in this age of government, looking

54:19

into what they're doing and seeing does this look like

54:22

a monopoly in ways that

54:24

hurt consumers or hold things back. Apple

54:27

needs to show that they're leading and that they're not

54:29

just being complacent in the next big thing. On lots

54:31

of levels, I think they need to show governments that,

54:33

they need to show probably the stock market that, and

54:36

they definitely need to show their customers that. They

54:38

don't need us all to be looking over

54:40

in Google Land saying, hmm, they're

54:43

doing a lot of good things with AI when

54:45

they're on those Google models these days. Maybe I

54:47

should switch to Android for this coming fall or

54:49

whatever. They don't need anybody thinking

54:51

that way, so they need to show everything's

54:54

fine over here in Apple Land in this

54:56

new age of AI. Please stay here, it's

54:58

nice and comfortable, don't switch. We

55:01

have features too, you can be happy with our version.

55:03

And I

55:06

think there's enough reporting

55:08

now around this narrative that it's

55:10

probably true. They were probably caught

55:12

off guard with the rise of

55:14

new L and base techniques. They

55:16

did probably decide very

55:18

late, possibly like a year ago, to

55:21

hey, we should probably start investing

55:23

heavily in this. We kind of missed the boat on

55:25

this. It does seem like

55:27

there might have been some kind of, Bill

55:29

Gates, we missed the internet kind of realization there. But

55:33

we'll see how quickly they can actually

55:35

take action here. I think on

55:37

some of the big stuff that Google and

55:40

Microsoft are demonstrating, I think Apple's gonna be

55:42

behind for a while because that stuff that

55:44

takes years to develop, it takes priorities and

55:46

skills and connections that Apple doesn't have. Not

55:49

to say they can't have it, but they currently choose not

55:52

to. So we'll

55:54

see what happens. It took

55:56

a lot of problems and

55:58

rot to get where we

56:00

are today with Siri. Have

56:03

they fixed those problems? Have they cleaned

56:05

up that rot? Have they

56:07

even realized and admitted to themselves that they

56:09

needed to do that? We don't

56:11

know yet. Or they're just pouring AI sauce

56:13

over the rot. Right, I mean that is

56:15

very possibly what we're gonna see. I

56:17

have a feeling, I think what we're going to see, I

56:20

think it's going to disappoint a lot

56:22

of people for not being enough AI

56:24

sauce, but I

56:27

don't really care that much if we don't get all

56:29

of our AI wish list items in

56:31

two weeks. What I want to

56:33

see is have they turned the ship

56:35

around? Have they actually realized current

56:38

Siri is garbage? Have they actually

56:40

started moving in a

56:42

better direction with that? Or are

56:45

they kind of half asking this and

56:47

being complacent and thinking, what do you mean Siri

56:49

is the best voice system? We are the most

56:51

private, et cetera. If they start going

56:54

in that direction, kind of defending where they

56:56

already are and suggesting that we don't need

56:58

anything better than this, I will be

57:00

concerned. But what I'm

57:02

looking for is not for them to solve every single

57:04

problem in two weeks. That's unrealistic. I want to just

57:06

see are they going in the right

57:08

direction? And so if we get a preview of this

57:11

cool app AI interaction

57:13

based system that only

57:15

works in Apple's apps and only comes next spring or whatever,

57:18

if we get a preview of that and it's really

57:20

cool and it shows we will be able to do

57:22

that in our app, maybe in iOS 19 next year,

57:24

that would be great. I

57:26

will be happy with that. I'll be a

57:28

little upset that it didn't come sooner, but

57:30

hey, as long as good stuff is coming

57:32

soon, I can be patient and wait,

57:34

just like Apple is often a very patient company when it

57:36

comes to this stuff. I just want

57:38

to make sure that what we see are

57:40

signs that they're going in the

57:43

right direction, not just sitting back and hoping

57:46

that we're all going to start talking about

57:48

Division Pro again and stop looking about AI

57:50

features. When it comes

57:52

to these sort of inside Apple things,

57:54

especially when they cite specific executives, the

57:56

reliability is never great, right? Because this

57:59

is a game. telephone, people have grudges,

58:01

who knows what actually goes on. But

58:03

the two characterizations

58:06

of Apple's

58:08

decision to go all

58:11

in on AI this year exemplify

58:14

kind of the worst case scenario as far as

58:17

I'm concerned. I don't know if they're true, they

58:19

could be total BS. But here are the two

58:21

things. One, and we think we've mentioned this snarkily

58:23

on for shows that there's various rumors and supposed

58:27

tales from the inside saying, recently,

58:29

as a year ago or a year or a

58:32

half ago, important Apple executives who

58:34

are often named saw a

58:36

chat GPT and that made them realize

58:38

Siri sucks. And that's depressing to say,

58:40

how did you not know? That's what

58:43

it took for you to realize Siri sucks. I

58:47

hope that I really hope to God that's

58:49

not what it took. We hope

58:51

that's not true. We hope that it's just like

58:53

someone sort of extrapolating or exaggerating

58:57

a story they heard or whatever, we

58:59

would hope that Apple has known Siri as a

59:01

problem for a long time. And chat GPT was

59:03

just really like the straw that broke with Campbell's

59:05

back. But it's an unflattering story about Apple, which

59:07

is again, maybe widespread. And the

59:09

second unflattering story related to this is that

59:11

I think Federighi was named here.

59:13

It was like a mandate a year ago

59:16

that every team under Federighi has

59:18

to add some AI feature this

59:21

year. And that is an unflattering

59:23

story, but that's exactly what big stupid corporations do.

59:25

They're like, I don't know, but there's some big

59:27

thing. I don't care what you're doing team. There's

59:29

a mandate from on high that says every team

59:31

has to add some AI thing. I don't care

59:33

what it is, but it better say AI. That's

59:35

not the way to make a good product. And

59:38

this happens. This has happened frequently in

59:40

Apple's history. We know with more certainty that like,

59:43

for example, when a new a new

59:45

macOS feature back when macOS was

59:47

important, a new feature would come

59:49

out and Apple would mandate, you

59:52

know, all the teams have to now like add some feature

59:54

that takes advantage of spotlight or something. You know what I

59:57

mean? And teams would complain. They were like, we were in

59:59

the middle of. of making our application better, but then

1:00:01

a mandate came from on high that stop

1:00:03

what you're doing and make sure you carve out room

1:00:05

in this release to add like a spotlight powered feature

1:00:08

or something. I'm not getting a great example, but

1:00:10

like that is disruptive to teams

1:00:12

who are working. Maybe the spotlight, adding a

1:00:14

spotlight feature is not important for whatever application

1:00:16

that, you know, it's not important for the

1:00:18

terminal application or whatever. Not that there's more

1:00:20

than a third of a developer working on

1:00:22

that, but like telling teams, you just

1:00:24

have to add something with AI. That's

1:00:27

not vision. That's not leadership. That's just

1:00:29

like buzzword exists. We

1:00:32

want to be able to say something about that

1:00:34

buzzword, therefore a sacrifice part or all of your

1:00:36

schedule to figuring out what you

1:00:38

can do that uses AI. It's

1:00:41

the opposite of the Apple philosophy. Instead of figuring

1:00:43

out like what will help people

1:00:45

and then doing that, it's like we've

1:00:47

pre-decided that AI is good

1:00:49

in and of itself. You figure out something

1:00:52

to do with it so you can fulfill

1:00:54

that. And again, if it's an unflattering story,

1:00:56

it's totally unconfirmed, possibly manufactured

1:00:58

or made up by someone with a pessimistic view

1:01:00

of Apple or whatever, and we just really hope

1:01:02

it's not true. But those are the only two

1:01:04

things that I've heard from inside Apple, and I

1:01:06

just hope it's like, again, people with grudges or

1:01:08

people with a dim view of what's actually going

1:01:11

on, because you would hope that Apple is more

1:01:13

thoughtful. And honestly, I do think Apple has been

1:01:15

more thoughtful. I think the chat GPT thing, like I said,

1:01:17

is not that they didn't know that Siri was bad. It's just

1:01:19

this really pushed them over the edge to say, this is the

1:01:21

year we really have to do something, which, you know, fair enough,

1:01:24

because they've been trying to solve it for

1:01:26

years and failing, and it's a big complicated

1:01:28

organization, yada, yada. But I don't need

1:01:31

AI sauce to be poured all over

1:01:33

everything in all

1:01:36

their operating systems. I just need it used where it

1:01:38

can do the most good. And part of that is,

1:01:40

yeah, look at where other people have done things and

1:01:42

that people like, like erasing people from photos, they should

1:01:44

be doing that, right? But in

1:01:46

other places, you know, manufacturing emoji,

1:01:48

the messages team said, how are we gonna add

1:01:51

AI? You know, I guess summary is good, right?

1:01:53

Thumbs up on that. Someone

1:01:55

has the idea to manufacture emoji, maybe

1:01:57

have those people work on like, messages

1:02:00

in the iCloud sync improvements or something,

1:02:02

but... Or search improvements. Yeah,

1:02:04

there you go. Or archiving improvements. I

1:02:06

would set priorities differently, let's say, but

1:02:09

what can you do? What

1:02:12

I really want to see, too, as a developer, I mean

1:02:14

obviously I'm a little bit biased here, but I think it

1:02:17

would be better for the whole ecosystem, too. As

1:02:20

Apple adds AI features, they

1:02:22

have a massive advantage that

1:02:24

they have really good silicon

1:02:28

and their products to do local processing. And

1:02:31

historically, they create really good APIs.

1:02:34

We nitpick here and there, like, oh,

1:02:36

something is under-documented or whatever, or watch

1:02:38

connectivity sucks, but for the most part,

1:02:41

and it does, but for the most

1:02:43

part, Apple's APIs are world-class. They're

1:02:45

really good most of the time. They

1:02:48

are extremely powerful. It's

1:02:50

very, very difficult to find better APIs for a

1:02:52

lot of things than what Apple provides. They're

1:02:55

excellent, and they allow us as

1:02:57

developers to make really great apps that

1:02:59

do really powerful things pretty easily. And

1:03:02

one of the reasons that they're able to do that is

1:03:04

because their local device hardware

1:03:06

bar is just so high.

1:03:09

They have all sorts of great APIs for

1:03:11

things like ML processing, which is going to

1:03:14

be presumably re-banded AI processing a lot of

1:03:16

this stuff. They have all sorts

1:03:18

of high processing power things that you can

1:03:20

just drop in in your app with not

1:03:23

much effort, and all of a sudden now you have

1:03:25

this audio training model or whatever.

1:03:28

What I want is as Apple

1:03:31

digs into modern AI

1:03:33

stuff, give that to us. Make

1:03:36

that available for free with

1:03:38

no limits right on

1:03:40

the device. That's something that not every competitor

1:03:42

can offer because they don't have, first of

1:03:45

all, the very high bar. Not every Android

1:03:47

phone is going to be able to do

1:03:49

this kind of stuff because not every Android

1:03:51

phone has the right processor. Microsoft

1:04:00

is in a situation where Intel and AMD

1:04:02

are just now going to be getting out

1:04:04

chips that can do 40 tops or whatever,

1:04:07

and they have that one Snapdragon chip. But

1:04:09

every Apple Silicon thing Apple has sold in

1:04:11

the past several years has had a pretty

1:04:13

powerful neural processing unit in addition to all

1:04:15

sorts of other hardware units in the SOC

1:04:17

for other tasks. So they're way ahead of

1:04:20

the game on hardware. And

1:04:22

what I want to see from them is not

1:04:24

only make that hardware available to developers – I

1:04:27

mean most of it already is – give

1:04:30

us the models. Have the

1:04:32

models for many common tasks built

1:04:34

into iOS and let us just call

1:04:36

them when our apps are running and

1:04:39

just use them unlimited for

1:04:41

free. That is

1:04:43

how you make an entire ecosystem of

1:04:45

awesome apps that run on your platform

1:04:47

and keep people locked in. So

1:04:50

there is no reason for them not to offer

1:04:52

this. For instance, one of the things I want

1:04:55

is build in a really good

1:04:57

transcription model. There is

1:04:59

one. There has been a speech

1:05:01

recognition API in

1:05:04

iOS for something like six or seven years.

1:05:06

It's not that new. It's

1:05:09

not very good and it's pretty limited. And it's

1:05:11

like in certain contexts I think they would send

1:05:13

it to the cloud because it would limit you to like 60

1:05:15

seconds of transcription at a time and stuff like that. But

1:05:18

now we're past that. Now we can do all that on device.

1:05:20

I don't want to have to ship OpenAI

1:05:23

Whisper and compile it exactly right for Apple

1:05:25

Silicon and make sure it's optimized right using

1:05:27

Whisper CPP and keep updating it every time

1:05:29

it gets updated and have my app have

1:05:31

to download this gig and a half file

1:05:33

for the model. Build that kind

1:05:35

of stuff in and just let us call it and

1:05:38

let us use it like we can use any other hardware resource. So

1:05:41

obviously there would be some limits on like background execution

1:05:43

burning your CPU and stuff, but like build

1:05:45

that stuff in and let everyone use it

1:05:47

for free unlimited. That is how

1:05:49

you make the next generation of awesome apps

1:05:52

on your platform. That's what I'm looking for.

1:05:54

I'm looking for A, give me a sign

1:05:56

that you're taking this new technique

1:05:58

of computing seriously. in

1:06:01

what you're building in as features to your OS. Also,

1:06:05

don't just keep it for yourselves. Build

1:06:07

APIs for it and let us use it in

1:06:09

the most unlimited way you can possibly have with

1:06:12

the hardware that you have. Obviously, if things go to

1:06:14

the cloud, yeah, you can

1:06:16

limit those, keep them to yourself or meet with them or whatever.

1:06:19

Let us do as much as we can locally. That's

1:06:21

what your devices are awesome at. Let

1:06:24

us use it. There are so many people out there, there

1:06:26

are so many app developers out there that

1:06:29

go through the process of figuring out how to

1:06:31

train your own model or integrate someone else's model

1:06:33

using these weird Python tools that we don't know

1:06:35

how to use. Just build

1:06:37

it in, let us use it. That would be

1:06:39

amazing. They do have

1:06:41

a track record of falling on both sides

1:06:43

of this with different decisions. This could go

1:06:45

either way. I really hope they go the

1:06:47

direction of here's a bunch of

1:06:50

awesome built-in models for everyone to use. The

1:06:53

limitations you're going to run into is not going to probably

1:06:55

be the limitations on

1:06:57

running in the background using CPU, stuff like

1:06:59

that. The rumor is for

1:07:02

this and what Microsoft has actually announced at

1:07:04

Build is that both companies

1:07:06

are essentially going to give access to models

1:07:08

but through abstracted APIs where the caller doesn't

1:07:10

even know or possibly even

1:07:12

get to choose whether it runs locally or

1:07:14

in the cloud so Apple can change that

1:07:17

decision on the fly in the new versions.

1:07:19

Microsoft advertised that at Build, hey,

1:07:21

you just use these frameworks and you get these features.

1:07:23

Don't worry about where it runs. We'll run it locally if

1:07:26

that's best. We'll run it in the cloud if that's best.

1:07:28

We'll mix and match. You don't have to know. It's all

1:07:30

abstracted from you. I would imagine any APIs like this that

1:07:33

Apple offers are going to do the same thing and that is the rumor that Apple's going to

1:07:35

have APIs

1:07:37

that give access to quote-unquote AI where it's abstracted,

1:07:39

where you don't have to know or care if

1:07:41

it's running locally or in the cloud. They'll do

1:07:43

whatever is best. Pick your feature. You just mentioned

1:07:45

transcription. That's a perfect example of like, there's

1:07:48

a new transcription API and it's better than the

1:07:50

old one and sometimes it runs locally and sometimes

1:07:52

it's not something but your app doesn't have to

1:07:54

worry. You just call this new abstracted API and

1:07:56

we will do the best thing we can do

1:07:58

and as phones get more RAM and so on

1:08:00

and so forth. It'll just get better and better, but

1:08:03

you call the same API the whole time. The question

1:08:05

is, what are those specific APIs? Microsoft announced a bunch

1:08:07

of build. They have 40 AI models

1:08:09

inside Windows, right? And I think Apple will

1:08:11

ship a bunch of models with iOS and

1:08:14

with macOS, hopefully, if they remember that it

1:08:16

exists, with iPadOS. And they

1:08:18

will have frameworks fronting them, but for what? Are

1:08:20

they going to have transcription? Are they just going

1:08:22

to have summarization, translation? There's so many different things

1:08:25

that they can do. So I think that's the

1:08:27

question that WWDC is, I guarantee they're

1:08:29

going to do that. They're going to ship models, they're going to

1:08:31

provide access to the sensor framework, but for

1:08:33

doing what? And you hope it's

1:08:36

transcription for the purposes of Overcast, but there

1:08:38

are many things they could choose to do.

1:08:40

They could even take existing APIs and just

1:08:42

say, hey, by the way, it's the same

1:08:44

exact API as it was before, but now

1:08:46

behind the scenes, it's implemented totally differently and

1:08:48

we use LLMs for it. So

1:08:50

we'll see. Can I have one

1:08:52

small request also on that, though?

1:08:55

Last time I checked, the speech recognition

1:08:57

API required microphone access to be granted.

1:09:00

Can that please not be the case if you're not

1:09:02

using the microphone, for God's sake? You can file that.

1:09:05

I filed the thing where I need screen recording permission to find

1:09:07

out where the desktop picture is. I mean, that kind of makes

1:09:10

sense in one way, but another way, I don't want to record

1:09:12

people's screens. I mean, what

1:09:14

could you possibly want transcription for if not

1:09:16

the things that you're speaking right now, Marco?

1:09:18

I don't understand. There is no other spoken

1:09:20

content. There's no other source of audio than

1:09:22

the microphone, Marco. I don't know if you're

1:09:24

dealing with that. There is no other spoken

1:09:26

content. Where else would audio come from? Yeah,

1:09:28

that's the thing, too. I hope, as

1:09:30

Apple hopefully adds

1:09:33

or expands the APIs to access all this

1:09:35

cool new stuff that I hope they're giving

1:09:37

us, I hope they

1:09:39

do it in broad ways. The

1:09:43

way Apple does things, I

1:09:46

think in a way that

1:09:48

fails power users and developers,

1:09:50

is sometimes they'll have some

1:09:52

kind of lockdown area of the system. Like,

1:09:54

all right, we're going to add an extension

1:09:56

or an entitlement for only this one very

1:09:58

narrow use case to... Maybe try to

1:10:01

use this this little ability that we're gonna,

1:10:03

we're gonna be a modest little tiny hole

1:10:05

in our in our fence here and just

1:10:07

permits is very narrow use case and as

1:10:10

a result is usable by like zero to

1:10:12

one. Apps are out there ever were and

1:10:14

and sometimes they they make larger a p

1:10:16

eyes that are useful for every one and

1:10:19

and can be used much more broadly and

1:10:21

like. If you if you

1:10:23

only do a former approach and not the ladder.

1:10:26

You. Never find new great applications like

1:10:28

the The. The market never really

1:10:30

breaks out of the the general

1:10:33

capabilities that Apple was able to

1:10:35

consider as both existing and as

1:10:37

important enough to warrant them gracing

1:10:39

us with a doorway to let

1:10:41

us do it. Whereas if you

1:10:43

make things more general purpose more

1:10:45

broad has fewer restrictions, let people

1:10:48

just kind of like. Use.

1:10:50

More general purpose tools. Ah.

1:10:53

You get apps that that Apple not only

1:10:55

in a benefits from in the sense that.

1:10:58

More. People want to use their platforms but it

1:11:00

also gives them ideas and what to sherlock for

1:11:02

the next releases. You get stuff like Quicksilver on

1:11:04

the Mac because of a dropbox on the Mac

1:11:06

elect these are all apps that like they they

1:11:09

cut to be manage of like. System.

1:11:11

Background stuff You get things like switch

1:11:13

as if you know you get upset

1:11:15

that at the city usually can only

1:11:18

exist on the Mac and are not

1:11:20

possible on i owe us see if

1:11:22

Eat but you need to like enabled

1:11:24

his Power User utilities and power user

1:11:27

features Eats enable them to exist in

1:11:29

ways that you as Apple didn't foresee

1:11:31

as. Like that with the they

1:11:33

didn't have to make and eighty I to

1:11:36

let dropbox in a badge. the file things

1:11:38

at first they get hacked. it's work like

1:11:40

through that hacks they already had and and

1:11:42

the waiter mid may be I to make

1:11:44

it better that I think is still kind

1:11:46

of in a trance synthetic of so arse.

1:11:48

yeah exactly or but. I hope

1:11:50

as their as their diving into all

1:11:52

this new ai stuff. I.

1:11:54

hope not only are i were saying earlier

1:11:56

that the that they have a p eyes

1:11:58

for us to use a functionized built-in

1:12:01

system model that we don't have to make

1:12:03

and train and ship ourselves. But

1:12:05

also, I really hope that they

1:12:07

allow access to them in broad

1:12:09

ways. Now I'm not talking about

1:12:11

like don't let me just burn everyone's

1:12:13

battery down like crazy, which by the way, there's

1:12:16

already an API for background

1:12:18

task management where you can

1:12:20

specify, I believe it's called a

1:12:22

background processing task. This is

1:12:24

a type that you can say that you can tell the

1:12:26

system when you have

1:12:28

a chance, when you are plugged

1:12:30

in and charging and maybe even

1:12:32

on Wi-Fi, call me in

1:12:34

the background, wake me up and let me do a

1:12:36

task with no throttling. And

1:12:39

it will do that. And if the person unplugged their

1:12:41

phone, it will terminate it and whatever. But there

1:12:44

are ways, there are opportunities for apps

1:12:46

to do background processing on iOS that

1:12:49

don't burn your battery down, but still allow them

1:12:51

to use the power of the hardware in the

1:12:54

background if they want to. You can try not

1:12:56

to wait until overnight to do it, but that

1:12:58

option is there. That API is already there. So

1:13:01

give us broad access. Let us do

1:13:04

whatever we want within the

1:13:06

existing battery and

1:13:08

power limitations you already enforce,

1:13:10

slash grant us. Let

1:13:12

us do whatever we want with this stuff. That

1:13:14

will enable great apps to be made. It

1:13:17

will enable everyone else on your platforms

1:13:20

to make the apps and features that you

1:13:22

won't make. And that

1:13:24

will both enhance your platforms for everybody

1:13:27

and whatever does take off, either

1:13:30

it will be something that's a little bit tricky,

1:13:33

like image generation that you won't then have to make

1:13:35

and take the liability of, and

1:13:37

or it will be successful and

1:13:40

you'll be able to then copy it for your next stuff and share it

1:13:42

with you in the next release. So

1:13:44

that's a win-win for Apple, and the

1:13:46

more of these features rely on that

1:13:48

local hardware and are

1:13:50

not based on cloud stuff, that

1:13:52

benefits their privacy strategy, that benefits

1:13:54

their hardware strategy, and that keeps

1:13:56

people locked into iPhones. every

1:14:00

reason for Apple to do it this way. Give

1:14:02

us a bunch of models and open them up as much as

1:14:04

possible to our apps to use. This

1:14:06

is a philosophical change that Apple has

1:14:09

not been on board with in recent decades that

1:14:11

we've all complained about multiple times is the idea

1:14:13

that good ideas can come from places other than

1:14:15

Apple. And Apple will say that they believe that

1:14:17

and support it, but not to the degree where

1:14:20

they will do what you just asked. Which is

1:14:22

open up APIs to allow third parties to do

1:14:24

things that historically in the past several decades Apple

1:14:26

has said not only Apple can even attempt that.

1:14:28

Like window management that I always complain about the

1:14:30

third parties could not have implemented stage manager. We

1:14:32

had to wait for Apple to think it had

1:14:34

an idea about window management and then it implemented

1:14:36

stage manager. And if you don't like it, wait

1:14:39

another five years for Apple to have another idea.

1:14:41

But no, we're not going to provide you APIs

1:14:43

to do that because no good ideas can come

1:14:45

from third parties. They're too dangerous. You can't have

1:14:47

this power, so on and so forth. The older

1:14:49

Apple, whether intentionally or not, essentially gave

1:14:51

enough free reign to APIs for tons of

1:14:53

good ideas to come out of the third

1:14:55

party developer community, which Apple then incorporated into

1:14:57

its operating system, right? And that was a

1:14:59

system that worked. And we didn't call it

1:15:01

Sherlocking back then. We just, it was just

1:15:03

the cycle. You know, Sherlock was an egregious

1:15:05

thing where they copied a particular app very

1:15:07

specifically in ways that were obvious that they were

1:15:09

copying it or whatever. But like giving

1:15:12

APIs where third parties can have ideas

1:15:14

and implement them that Apple can learn

1:15:16

from was how the first 20

1:15:18

to 30 years of the Mac, or

1:15:20

maybe I'm getting years wrong, or the first the

1:15:23

early part of the Mac. That was how the

1:15:25

platform evolved. To give a modern

1:15:27

example, how did Twitter evolve in the early

1:15:29

days? By having good ideas

1:15:31

happen in the third party world. Good

1:15:34

ideas like the concept of a retweet and

1:15:36

using at to mention somebody and the word

1:15:38

tweet all came from third parties. Current

1:15:41

Apple thinks that there are certain classes of

1:15:43

ideas that can only come from

1:15:45

Apple. And so they closed themselves

1:15:47

off to lots of good things. So

1:15:49

like you third party developers shouldn't have

1:15:51

an API powerful enough to do this.

1:15:54

When we, Apple, eventually five years from

1:15:56

now, come up with a good idea

1:15:58

for doing something with this. implemented

1:16:00

but you can't have those API so

1:16:02

audio hijack on iPad Apple

1:16:05

will get around to it eventually but it's not like we're

1:16:07

gonna let a third party do that stage manager oh third

1:16:09

party do you have ideas about window management sorry that's too

1:16:11

dangerous for you we can't give you that kind of control

1:16:13

it limits us too much it limits Apple too much because

1:16:15

what if we have an idea we locked

1:16:18

ourselves into a bunch of API's that are being used

1:16:20

by third party applications we don't want to do that

1:16:22

it's safer to just you know and the thing is

1:16:24

when this philosophy first rolled out with sort of the

1:16:26

Mac OS 10 error it was it's

1:16:29

like a pendulum right it was a relief because

1:16:31

Apple had swung too far in the other direction

1:16:33

where they would give API's third parties get locked

1:16:35

into them because popular third party products would use

1:16:38

them and Apple would be constrained into what it

1:16:40

could do and so they swung hard in the

1:16:42

other direction said you know what we're

1:16:44

not giving anything like that to third parties

1:16:47

we're gonna keep it all real close to

1:16:49

the vest be very very conservative close off

1:16:51

innovation of third party world to give Apple

1:16:53

itself more flexibility to innovate and

1:16:56

evolve and the pendulum had swung too far

1:16:58

in the other direction but now it's so

1:17:00

far on the opposite side of things that

1:17:02

we are out here suffering from you know

1:17:05

the lack of Apple on third parties to innovate

1:17:07

and AI is the newest front of that because

1:17:09

we see this explosion of you know we described

1:17:12

as throwing spaghetti against the wall but another

1:17:14

way to describe it is exuberant innovation

1:17:16

enthusiastic innovation most that stuff's not gonna work

1:17:18

out but some of it is the more

1:17:20

you give third parties flexibility to do that

1:17:22

the better ideas you'll get so I just

1:17:25

you know there's either

1:17:27

extreme is wrong and we are currently in the Apple

1:17:29

world at one extreme so I hope it starts swinging

1:17:31

back the other direction we

1:17:34

are brought to you this episode by delete me you

1:17:37

ever wonder how much your personal info

1:17:39

is out there for anyone to really

1:17:41

easily find on the internet it's way

1:17:43

more than you think try it search

1:17:46

for yourself and you'll see like how

1:17:48

quickly and easily anybody can find out

1:17:50

all your information name address phone numbers

1:17:52

your social security number sometimes even

1:17:55

all this and all this too about your family

1:17:57

members which is extra concerning so all this is

1:17:59

being filed by data brokers, openly traded

1:18:01

and sold online. Anybody can really easily

1:18:03

get these details from you. So

1:18:06

protect your privacy with delete

1:18:08

me. I actually found delete

1:18:10

me myself when I was looking to reduce some of

1:18:13

my information out there a few years ago before they

1:18:15

were a sponsor. And I did some quick research, found

1:18:17

them and I chose to use them myself because

1:18:20

there's a lot out there. As

1:18:22

I said, it's super easy to find and you can't

1:18:24

get everything, every single possible detail but you can get

1:18:26

a lot of it. And that's why services like delete

1:18:28

me are great and I chose them and I've been

1:18:30

very happy with them. So delete me, find

1:18:32

and remove any personal information that you don't

1:18:34

want online. You tell them what you want

1:18:36

gone, they make it gone. And

1:18:39

they also, they're an ongoing subscription service. So

1:18:41

they not only remove your info from the

1:18:43

largest people search databases on the web but

1:18:45

they also then keep monitoring it

1:18:47

and they keep updating and they keep making sure

1:18:49

that your information is still gone. So you sign

1:18:51

up and provide them with exactly what you want

1:18:54

deleted and they take it from there.

1:18:56

So they have their personalized privacy reports showing what they found,

1:18:58

where they found it, what they were able to remove. And

1:19:01

so it does all the hard work for you

1:19:03

of wiping your and your family's personal info off

1:19:06

the web. So check that today,

1:19:08

take control of your data and keep your

1:19:10

private life private by signing up for delete

1:19:12

me. Now at a special

1:19:15

discount for our listeners, get 20%

1:19:17

off your delete me plan when

1:19:19

you go to joindeleteme.com/ATP and use

1:19:22

form a code ATP at checkout.

1:19:24

The only way to get 20%

1:19:26

off, go to joindeleteme.com/ATP, code ATP

1:19:29

at checkout. That's joindeleteme.com/ATP, code ATP.

1:19:31

Thank you to delete me for

1:19:33

sponsoring our show. All

1:19:39

right, so I see here in the show

1:19:41

notes, LLM check-in. And so I

1:19:44

guess, John, you'd like to take us to the world

1:19:46

to give us a little situation report. Yeah,

1:19:48

LLMs. They're weird. There's

1:19:50

been a lot of stuff online trying

1:19:53

to explain to people how they work.

1:19:56

They're a type of technology where when

1:19:58

people look at them, they're guess about

1:20:00

what they are and how they work is very often

1:20:02

wrong because part of their whole deal is the reason

1:20:04

people are so fascinated by them is because they can

1:20:06

essentially fool you. This

1:20:10

is related to a recent

1:20:12

story. I think we panted out in past shows, but

1:20:14

you probably have seen it in the news, where Google

1:20:16

is replacing their thing that gives you the summary of

1:20:18

the top of their search results. They're replacing that with

1:20:21

an AI-powered one. They poured AI sauce on that. It's

1:20:23

been doing some silly things like suggesting that you

1:20:26

eat one rock per day and suggesting that to

1:20:28

keep the cheese from sliding off your pizza you

1:20:30

should add glue. That's been pulling

1:20:32

things from the onion and presenting them

1:20:34

as straightforward things and not parody. Someone

1:20:37

writes a snarky post on Reddit and Google

1:20:39

pulls it out and spits it back to the user. Ha

1:20:42

ha, isn't that funny? AI is so dumb. Then

1:20:45

Google has been trying to fix these

1:20:47

manually and work on it. I

1:20:50

bet people who see this are like, oh, AI is dumb, but

1:20:52

it will get better or whatever. Another

1:20:55

thought that people often have and express is, why

1:20:58

doesn't Google just tell its AI

1:21:00

model that you shouldn't eat glue? The

1:21:03

glue doesn't go on pizza. Why doesn't it

1:21:05

just tell it people shouldn't eat rocks? Like

1:21:08

zero is the correct amount of rocks to eat

1:21:10

per day. Why doesn't

1:21:12

it just do that? I know that, okay, well,

1:21:14

it's new, whatever, it'll get better. Because

1:21:18

of the mental model people have about

1:21:20

LLMs, it seems so silly that they would

1:21:22

give an answer so dumb and

1:21:24

it's like, well, why doesn't Google

1:21:26

just correct it? These aren't nuanced corrections.

1:21:29

Don't eat rocks. It

1:21:32

seems so simple. I think part

1:21:35

of it is because that's not how

1:21:37

LLMs work. I'll link again to these three blue,

1:21:39

one brown neural network things or whatever. I know

1:21:41

it's complicated or whatever, but if you just keep

1:21:43

watching these videos over and over again as I

1:21:45

have, don't you start to sink in

1:21:47

what they are. The analogy you gave on erectus

1:21:49

ages ago is a pachinko machine where you drop

1:21:51

a ball at the top of this big, giant

1:21:53

grid of pins and it bounces off pins and

1:21:55

eventually lands at a particular place. The

1:21:58

thing I was trying to express by that is... But the way

1:22:00

LMS work is they take an input, which

1:22:02

is whatever it may be, but in this case, let's just

1:22:04

say some text. And it bounces

1:22:07

around inside doing a bunch of math, and

1:22:09

some text comes out the other end. And you could do it

1:22:11

with images and other things or whatever. But it's just a bunch

1:22:13

of matrix math, and

1:22:15

a thing comes out the other end. And it's

1:22:17

stateless. The thing you're putting

1:22:19

it into, the machine, the LLM, has been

1:22:22

trained. And the training is to essentially set

1:22:25

a bunch of things called weights, which are a bunch

1:22:27

of numbers inside the model. It's just a huge number

1:22:29

of these. And the magic

1:22:31

is, well, where did those numbers come from? They came

1:22:33

from training. And you can look at the video and

1:22:35

see how this works. But it's like, we're going to

1:22:37

make this giant grid of pins and

1:22:40

just grind on it with a huge amount of

1:22:42

computing resources feeding the entire incident in. And

1:22:44

the result is just tons and tons

1:22:46

of numbers, tons

1:22:49

and tons of pins in the pachinko machine. And

1:22:51

after we've done that, we drop things on the top

1:22:53

of the machine. It bounces around. Stuff comes out. And

1:22:55

the quality of stuff comes out depends on what those

1:22:57

pins were, what those numbers were, what the weights were.

1:23:00

And the reason people get hung up on the gluing

1:23:02

pizza and eating rocks is just tell

1:23:05

it people shouldn't eat rocks. Is that hard

1:23:07

to teach it? That's like, well, you don't

1:23:09

understand. We do this training. And it's very

1:23:11

expensive. And it takes a long time. And

1:23:13

we produce this giant model. But once

1:23:15

you've got the model, you can

1:23:17

retrain and tweak and adjust or whatever.

1:23:20

But you can't just type to it,

1:23:22

oh, and by the way, people shouldn't

1:23:24

eat rocks. Because you drop it into

1:23:26

the top of the machine, and it would bounce out.

1:23:28

And so there's nothing that people might not understand from

1:23:30

first glance of how these things work is

1:23:33

when you're having a quote unquote conversation with chat CPT,

1:23:35

and you ask it a question, like how many rocks

1:23:37

should I eat per day? And it gives you an

1:23:39

answer. And so it gives you a bad answer. It

1:23:42

says you should eat one rock per day. You type back

1:23:44

to it. Actually, people shouldn't eat

1:23:46

rocks. If you eat rocks, it's really bad for you.

1:23:48

Bad things can happen to you. And

1:23:51

then it will reply to you, oh, I'm sorry. I made that

1:23:53

mistake. Blah, blah, blah, blah. What you don't realize is every time

1:23:55

you type something to chat

1:23:58

CPT, when you type a message, people don't know. shouldn't

1:24:00

eat rocks. That's just not what gets

1:24:02

sent to ChatTPT. What gets sent is the entire

1:24:04

conversation up to that point. Your first

1:24:06

question, ChatTPT's answer, your second question, every time

1:24:08

you type something new, the whole conversation gets

1:24:11

sent through. Because remember, ChatTPT doesn't have any,

1:24:13

I know there's a memory feature or whatever,

1:24:15

but the LOM itself is just a big

1:24:17

bucket of pins in a pachinko machine. You

1:24:19

drop something in and it comes out. So

1:24:22

your quote unquote conversation, all you're doing is

1:24:24

making the thing you're sending it bigger and

1:24:26

bigger each time. There's no back and forth.

1:24:28

There is, here's the entire conversation plus my

1:24:30

new thing, including what it answered before. Because

1:24:33

that influences what the next thing is

1:24:35

going to come out. In fact, when it's processing things, what it

1:24:37

does is it processes the entire input and picks the next word,

1:24:40

and then it throws everything back in and picks the next word,

1:24:42

and then picks the next word over and over again until it

1:24:44

gets all the words in the answer for your thing. That

1:24:47

is essentially stateless.

1:24:50

There is no thing that you can say people shouldn't

1:24:52

eat rocks. All you can do is put that in

1:24:54

your conversation, and then when the whole conversation goes back

1:24:56

in the top of the machine again, yeah, it's in

1:24:58

there and it influences the output according to the magic

1:25:00

of all the weights and everything. That

1:25:05

whole thing of like, okay, but how big is that?

1:25:07

What is that? I can send the whole conversation back

1:25:09

in. The length of stuff

1:25:11

you can stick into an LOM is called the

1:25:13

context window. If I have a conversation and it

1:25:15

goes wrong for thousands and thousands of words, at

1:25:17

a certain point, there's a limit in the size

1:25:20

of the input string. The input string to

1:25:22

LOMs used to be very small and I was

1:25:24

getting bigger and bigger. This is related to an

1:25:26

announcement Google had at their I.O. conference where

1:25:29

the CEO said, today we

1:25:31

are expanding the context window to

1:25:33

two million tokens. That's a

1:25:35

big number because most of the context of the context would

1:25:37

just start off at like 32K or whatever. Google is saying

1:25:40

we're expanding to two million tokens. We're making it available for

1:25:42

developers in private preview. It's amazing to look back and see

1:25:44

just how much progress we made in a few months. You

1:25:47

can see how the context window would be important because

1:25:49

if you wanted to, quote unquote, teach it that people

1:25:52

shouldn't eat rocks, what you'd want is to be able

1:25:54

to stay in a conversation. You

1:25:56

just told me I should eat one rock per day, but that's

1:25:58

really bad. Humans shouldn't eat rocks. And you

1:26:00

want to, quote unquote, remember that and,

1:26:03

quote unquote, learn that. But

1:26:05

the way I almost work, the only way I can remember

1:26:07

or learn that is either A, you train on new data

1:26:09

that influences the weights in the model, which is something you

1:26:11

can't do when you're typing to chat GPT. You're not changing

1:26:14

the weights in the model. You're just

1:26:16

sending things through an existing machine. Only OpenAI can

1:26:18

change those weights, right, by making a new model

1:26:20

or whatever, or modifying the existing one, right? Or

1:26:23

B, have that phrase be

1:26:25

part of the context window that

1:26:29

your thing is included in. Like the example I used to give with

1:26:31

Merlin was like, if you say, you

1:26:33

know, I have one brother and two, you

1:26:35

know, how many siblings do I have? And it says, I don't

1:26:37

know. And you tell it, I have one brother and two sisters.

1:26:39

And then you say, again, how many siblings do I have? And

1:26:41

it says, you have one brother and two sisters. Like, wow, it

1:26:43

learned it. No, because the input you put in was, how many

1:26:45

siblings do I have? I don't know. I have one

1:26:48

brother and two sisters. How many siblings do I have? That

1:26:50

was the input. The answer is in the input. So

1:26:52

you shouldn't be shocked when it says, you have one

1:26:54

brother. Hey, it learned that I have one brother. No,

1:26:56

the input contains the answer. It was part of the

1:26:59

context window. You didn't change any of the weights in

1:27:01

the model. You literally just gave it the answer, right?

1:27:03

So here's the final thing. At On Stage at Google

1:27:05

I.O., Sundar Pichai, Google CEO said, talking

1:27:08

about the two million context

1:27:10

window, this represents the next step in

1:27:12

our journey towards the ultimate

1:27:14

goal of infinite

1:27:16

context. And

1:27:19

this is the first time I've seen someone outline a

1:27:22

vision for how LLMs could

1:27:24

actually be taught things. Because

1:27:27

if the context window is infinite, what

1:27:29

that would mean is that you

1:27:31

could talk to an LLM over the course of

1:27:34

months, days, months, and years. Everything

1:27:37

you ever said to it would essentially be

1:27:39

sent as input in

1:27:42

its entirety, plus the new thing that you said

1:27:44

every time. So if you said six months

1:27:46

ago people shouldn't eat rocks, every time you

1:27:48

ask any question, part of the input

1:27:50

would be your question, plus everything you've ever said to

1:27:53

it, including the line that people shouldn't eat rocks. And

1:27:55

so when it answers you, it will, quote

1:27:57

unquote, remember that people shouldn't eat rocks. because

1:28:00

that was part of its input. I

1:28:04

don't think this is a way to make a reasonable kind

1:28:06

of mind that can learn things, but it

1:28:08

is the first vision I've heard of anyone

1:28:10

outlining how LLMs are not going to be

1:28:12

dumb. Because no matter how well you

1:28:14

train them on the big stew of stuff you're putting into them,

1:28:17

you can't teach them anything. They can't learn

1:28:19

through conversing with you. They can only learn

1:28:21

by being trained on new data and having

1:28:24

a new version of the model come out

1:28:26

or whatever. So we want them to work

1:28:28

like people where you can say, oh, silly

1:28:31

toddler, rocks are bad for you, don't

1:28:33

eat them and have it learn

1:28:35

that. If you have an infinite context

1:28:38

window, I mean, anytime you ask it anything, the

1:28:41

entire history of everything you've

1:28:43

ever said goes as input somehow. I

1:28:46

don't know if he said this is just kind of like a vision,

1:28:49

a conceptual vision or a practical

1:28:53

example of like, that's how we're going to do it. We're going

1:28:55

to have a stateless box of numbers

1:28:57

that we throw your input into, but we're

1:28:59

going to retain your input forever and ever

1:29:01

and ever. Anytime

1:29:03

you ask the stateless box of numbers anything, everything you've

1:29:05

ever said to it goes as input plus the new

1:29:07

thing that you said so that you can,

1:29:09

quote unquote, teach it things. The

1:29:11

fun part of like teachable AI or like

1:29:13

an actual sort of thing that you could

1:29:15

converse to is that you can just teach

1:29:17

it BS things. If you talk to an LLM and tell

1:29:19

it, actually, you should eat rocks. In fact, you should eat

1:29:22

really spiky rocks all the time. And

1:29:24

it says, OK, that's great. I'll remember that if you ever

1:29:26

asked me about what you should eat again. And six months

1:29:28

from now, you said, give me this recipe and it

1:29:30

includes rocks. It'll be like, well, as big as the input

1:29:32

was, you should eat spiky rocks, six months worth of text.

1:29:35

Give me a recipe for pizza and it includes rocks. You

1:29:38

could teach it to be less useful, like in the same

1:29:40

way of raising children. If you teach them bad things, they

1:29:42

will learn bad things. We

1:29:45

don't all have our own copy of chat GPT

1:29:47

or LMs or whatever. There's just one big stateless

1:29:49

blob that's instanced all over that our input is

1:29:52

going through. But if we

1:29:54

have our own infinite context window, then we have

1:29:56

we are essentially building our own

1:29:58

sort of knowledge. base within this LLM

1:30:02

through the barbaric brute force method

1:30:04

of sending it every piece of information we've ever

1:30:06

sent to it before with every new piece of

1:30:08

information. I just thought this was interesting because I've

1:30:10

been super down on LLMs because I just don't

1:30:13

see how they can ever be anything useful that

1:30:15

can be taught like the difference between fact and

1:30:17

fiction. Important

1:30:20

things that would make the thing more useful

1:30:22

are not possible because of the way LLMs

1:30:24

work. But if you give me an infinite

1:30:26

context window, at least then I can

1:30:29

over time try to mold my

1:30:32

little conversation with the LLM

1:30:35

towards something and maybe only have to correct it

1:30:37

once or twice when it makes mistakes so that

1:30:39

it will get better over time underneath

1:30:42

my own control. That

1:30:44

said, he is the CEO. I don't know if

1:30:46

he knows about this on a technical matter, an

1:30:48

infinite contact window sounds ridiculous to me. But I

1:30:50

just wanted to bring this up because it really

1:30:52

annoys me that LLMs essentially do not learn through

1:30:54

conversing with you even though everyone thinks they do

1:30:56

and annoys me when people get fake debt. Good

1:30:59

talk. I hear you. I

1:31:03

think watching these 3Blue1Brown videos, there's others that

1:31:05

are good too, but these are very, very

1:31:08

good. Even though they're

1:31:10

not fast in the sense that they're

1:31:12

not rushed, but they're fast in the

1:31:14

sense that a lot of ground is

1:31:16

covered very quickly, and I

1:31:18

think you're right that watching them like 2 or 3

1:31:21

times is what you probably need in order to get

1:31:23

this really understood. But

1:31:26

I strongly suggest if you

1:31:28

are even vaguely inclined for these

1:31:31

sorts of things, and if you're listening to the show,

1:31:33

I presume you are, it's worth checking it out. Just

1:31:35

understand kind of the broad strokes as to how these

1:31:37

things work. Thank you to

1:31:39

our sponsors this week. Fast mail and

1:31:41

delete me. Thanks to our members who

1:31:43

support us directly. You can join us at ATP.FM. Join.

1:31:46

One of the biggest member perks now

1:31:49

is we do this overtime segment where

1:31:51

every week, every episode, we have a

1:31:53

bonus topic that just didn't fit. We

1:31:56

couldn't get to it in the main episode. This

1:31:58

week, overtime is about the end. TikTok

1:32:00

ban in the US that

1:32:02

is currently working its way through the system.

1:32:05

We're going to be talking about that. You

1:32:07

can listen by joining at atb.fm slash join,

1:32:09

and we will talk to you next week.

1:32:30

And it was accidento. Oh,

1:32:32

it was accidento. You can

1:32:34

find the show notes at a-t-c-s-n-f. And

1:32:41

if your is your mathadon, you can

1:32:44

follow us at C-A-S-E-Y-L-I-S-S,

1:32:50

so that's

1:32:52

K-C-L-I-S-M-A-R-C-O-A-R-M-E-N-T, Marko

1:32:54

Armand. That's

1:32:58

I-R-A-C-U-S-S-C, R-I-S-A-T,

1:33:00

R-Q-S-R-E, That's actually

1:33:02

not a tool, that is a

1:33:06

tool.

1:33:09

E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E-E

1:33:12

NO HOOOONN. All

1:33:15

right, so in the section

1:33:17

of our internal show notes, where we

1:33:19

put after show ideas, and

1:33:26

it reads a couple things. One of

1:33:28

them is Marco's desk setup. And

1:33:31

you had privately teased something to the two

1:33:33

of us about this. And

1:33:35

I have been dying for this

1:33:37

to happen. And I feel like this

1:33:39

is the moment, we almost need an ATP overtime for

1:33:41

after show stuff. We finally got there.

1:33:44

Don't ruin my high, Marco. Tell me

1:33:46

what's going on. In

1:33:48

the new house, part of the condition of getting

1:33:51

this house was there was one room upstairs that

1:33:54

overlooks a view that you

1:33:56

can see. Long Island is full of canals,

1:33:58

because Long Island, everybody has both. I

1:34:00

don't have a boat, I don't want a boat, but there's canals everywhere. And

1:34:03

from one of these rooms upstairs, we can

1:34:05

see one of these canals. So

1:34:07

as a result of there being a canal, you know, at the

1:34:09

window, I can see the ducks

1:34:11

and the birds and other delightful things floating by

1:34:13

in the canal and hanging out. I can see

1:34:16

the ducks sit on the neighbor's lawn all folded

1:34:18

up in the rain. I can

1:34:20

see the rabbits jumping across my lawn. It's

1:34:22

a wonderful view. The only way

1:34:24

to enjoy this view is by

1:34:26

sitting basically in the middle of the room. So

1:34:30

when I was laying out my office

1:34:32

setup, the only way – like

1:34:34

the big deal breaker for my office

1:34:37

layout was the desk has to be

1:34:40

sticking out from one of the walls in the

1:34:42

middle of the room. It's

1:34:44

like a big capital E where the

1:34:46

walls are the outer walls of the E and the

1:34:48

one middle thing that sticks out of the E, that's

1:34:50

the desk. Like it's in the middle of the room.

1:34:53

Unlike people who mostly put their desks against the wall, I

1:34:55

mean, you're sitting at the desk, you would be facing the

1:34:57

wall. That's not what you're doing. I

1:34:59

am such that I'm facing – when

1:35:02

I'm sitting at my desk, the window is to my

1:35:04

right. And so I can just look out, I can

1:35:06

turn to the right and look out the window and

1:35:08

see the happy ducks sitting around and floating by. So

1:35:12

I want to do this. Now the problem with floating the desk in

1:35:14

the middle of the room is that first

1:35:17

of all, it imposes a

1:35:20

larger aesthetic burden on

1:35:22

the desk and the items on the desk

1:35:24

because you don't have the wall to hide

1:35:26

your sins. So one of the

1:35:28

things I did, one of the reasons why I

1:35:31

was drilling stuff into my desk recently is

1:35:33

that I got a desk that has kind of

1:35:35

like a compartment behind it for

1:35:37

all the wires to go into. So

1:35:40

behind my desk is clean wire management, not a

1:35:42

whole bunch of wires dangling there because again, it's

1:35:44

the middle of the room. Like half the room can see

1:35:46

the back of my desk. So things

1:35:49

have to be a little bit cleaner here to look

1:35:51

nice than at the beach where I

1:35:53

can just shove everything against the wall like everyone else does. As

1:35:55

a result, when I was choosing my stuff,

1:35:57

I was just going to put it in the middle of the

1:35:59

room. speaker setup i didn't have tall

1:36:02

speakers as an option uh...

1:36:04

my my preferred speakers that the k e

1:36:06

f uh... q one fifties right there those

1:36:09

are awesome speakers than they actually sound they found that

1:36:11

i still love them at the beach but they're really

1:36:13

big and boxy and tall and i have a little

1:36:15

desk in the middle of the room looked

1:36:18

ridiculous so i'm not i'm not going

1:36:20

to do that so i look at small

1:36:22

speaker options i wasn't sure what to do yet meanwhile uh...

1:36:25

we had a friend of the show who

1:36:28

was gracious enough to share a sonos

1:36:30

discount code with me i believe in

1:36:32

the fall was a while ago uh...

1:36:35

and at that time i know

1:36:37

knowing we were renovating the new house but you know

1:36:39

this is a good opportunity to get some discounted sonos

1:36:41

gear uh... i bought a

1:36:43

set for the future t.v.

1:36:46

i bought a soundbar and two era

1:36:48

three hundred to be used as surrounds

1:36:51