Free Form AI

Michael Berk (00:01.762)
Welcome back to another episode of Freeform AI. My name is Michael Berk and I do data engineering and machine learning at Databricks and I'm joined by my amazing co-host.

Ben (00:10.72)
I'm Ben Wilson and I figure out how to run Python code safely in sub-processes at Databricks.

Michael Berk (00:19.692)
not just for your own personal fun.

Ben (00:23.116)
I mean, sometimes, man, you never

Michael Berk (00:25.994)
That could be an interesting hobby, I'm not gonna lie. Subprocesses are kind of cool. Anyway, we're definitely not nerds here, I promise. But today's topic is actually going to be fricking awesome. Both Ben and I are excited about it. It's something we haven't heard discussed specifically from a tech lens. And hopefully it will inform how you guys go about bettering yourself. So.

The topic today is knowledge acquisitions and specifically knowledge acquisition for action. There's a lot of tropes of people who like knowledge for knowledge's sake, and we're not going to really touch on that. What we're instead going to be touching on is how you can use new information, put it in your brain and have it be used to produce useful outputs. So let's kick it off with why we're talking about this. My friend got me a book.

called Building a Second Brain by Tiago Forte. And I've been going through it and it's just like BS. it's such a, it's like not great. And a lot of the concepts are really important, but they're really redundant if you're just like a semi-competent professional. And I think they're over-trivialized and miss a lot of really key points. And so that got me thinking, Ben is theoretically good at his job. I'm curious how he approaches this.

so I threw down a list of questions and I'm very curious to hear Ben your take and also flesh out some of my opinions as well.

Ben (02:00.982)
Yeah, I mean, we're going to do round robin here. So we're going to ask a question and then we're both going to answer and we'll critique each other. Hopefully not as badly as Michael just get treats that guy's book, but yeah.

Michael Berk (02:13.464)
I'm only halfway through. It might have a really strong finale, but so far, don't read it. Yeah, Ben, you want to kick us off with the first question?

Ben (02:22.878)
Yeah, I mean, the first one we had on a list was how do you take notes?

Michael Berk (02:28.142)
Cool. I don't want to answer first. You go first.

Ben (02:33.14)
Yeah, I'll answer my own question. poorly. I think by nature, I'm somebody who is somewhat chaotic and disorganized with regards to, like planning things about like, I want to do these things. I really don't take a lot of, of notes about things that I'm going to be tackling in a particular day or week. What I do is project plans.

which is I have a list of anything that's of sufficient complexity. need to make sure that I know general scoping of how many hours I should devote to something, just so that I don't overburden myself. And then if I have context that I need to remember, I'll put that in that tracker. So I don't use anything fancy or anything. It's like usually Google docs and bullet point lists and links to things.

Like, I had this Slack conversation with somebody who requested this feature or this, you know, ticket that came in or something's totally busted. I got to fix this. And then I just work on prioritization based on that of what needs to get done first.

Michael Berk (03:47.8)
What does your note stock look like? Like what is in there?

Ben (03:51.352)
it's chaos. It's pure chaos. so it's usually like a header one of what the project is or the, just the general scope of some work stream that thing could extend a week or it could be six months. And then each of the different milestones that are in there of like, Hey, I got to ship this by this date.

Usually it's self-imposed, like something where I'm like, hey, I want to this out in two weeks from now. Here's the seven things I need to get done. And then I'll just create little check boxes next to them and take it off when I'm done. finding stuff in that doc is interesting. I do archive stuff though. So as things have like fully completed and I don't need to worry about it anymore, I'll put it into another like copy of that doc.

Michael Berk (04:50.606)
How often do you reread or leverage old notes?

Ben (04:54.744)
pretty much never. I do have another like protected encrypted note thing that I use that is just local to my, my, my work laptop that has not stuff like passwords, but, like just things that are very important that I need to do. And I don't want to exhaust mental energy and remembering all the steps involved. So it's like my own personal confluence page of.

Michael Berk (05:23.864)
Hmm.

Ben (05:25.016)
like, hey, here's how I released this package, because it's manual and I don't feel like building the automation for it, because I only do it like once every three months. So I have like the 17 steps of what I need to do just so I don't forget. That's about it.

Michael Berk (05:44.244)
it. All right. I'll answer the question before we get way too deep. I am really into notes, put simply, when I was in college, I used one note and basically had a bunch of folders, tried to organize by topic and really thought that knowledge acquisition meant how much you had on paper. So when I would read a book, I would take notes along the way, and then I would reread them, reorganize them.

et cetera. Then I was like, wait a second, I actually never reread my notes. So then I started a habit where every morning with my coffee, I would reread a randomly selected page of notes. That was cool to like, sure up and, I don't know, harden my knowledge acquisition, but it wasn't super useful. So I stopped doing that. Since then, I've started using Roam research, which is a

Basically, it's one big daily note page and it leverages tags to create a graph structure that links ideas. So if you look at your entire note structure, there will be actually a graph page that shows where sub pages are linked. And it's cool to see the connections. But I've had so much notes in there that it's starting to get slow. now I don't really, I don't know. I'm like not into notes at the moment. Beyond my working.

like job where I have a scratch document for each project. So that's how I take notes.

Ben (07:15.608)
Yeah, it was different for me like in college though. And every time that I've shifted careers, I know that I'm thinking about it. I do have this habit and it's very similar to yours, but I read somewhere and I don't remember where, apparently I didn't take a note of it. But the actual process of leveraging that part of your brain that requires you to

to form letters on paper. It helps to like solidify knowledge a little bit more. So I was always very studious about taking notes in class during lectures or labs. And I would write down just all the things that I thought were very important and never read them again. For some tests and stuff, yeah, I'd go back and study and try to remember that.

I've always been somebody who learns better by doing something, but having those notes kind of gave me that foundation for the knowledge that I think I needed to absorb. And I've done that, like I said, every time I've changed careers throughout my meta career, like moving to not just, I'm starting with a new company, but it's the same sort of thing that I was doing before. Yeah, that's important.

to do something like that. I don't feel like I would do that now. Like if I moved to a software engineering job at another place, I wouldn't have that same, I wouldn't feel like I needed to do that. But moving from the U S Navy to civilian like robotics engineering, I had no clue what the hell I was doing. So I was writing down all sorts of stuff like, just reading through this technical manual of like how this machine works.

I'm going to take a lot of notes. And sometimes I never read them, but it was just like that. That act of note taking helped me understand it a lot better.

Michael Berk (09:20.088)
So what you are potentially hinting at is that the action of taking notes is the value of the note, not looking it up again.

Ben (09:27.16)
Mm-hmm.

Yeah, for sure.

Michael Berk (09:31.086)
Do you think that is a fault in the note process?

Ben (09:38.178)
I don't know.

Michael Berk (09:40.908)
I don't either. That's like the whole thesis of this book, right? Is that notes are for retrieval. I'm on the same camp where the value that I get from notes is the initial writing it down, then it's generally in my brain and I really rarely go back to prior notes.

Ben (09:40.93)
No idea, man.

Ben (09:56.278)
Yeah, same.

Michael Berk (09:58.414)
That seems wrong. There's a second brain in your digital system. Shouldn't you be leveraging it so that there's less strain on recall for your human mind than it's deferred to the digital brain? Isn't that the concept, right?

Ben (10:17.058)
I don't find that it's like a strain for me to retrieve information. It's more of, I wouldn't say it's like intelligence or anything. It's just, if it's something, the way that I categorize information retrieval from my own personal bias is if you tell me a fact a week ago and maybe I write it down, I'm like, yeah.

Michael Berk (10:23.152)
Sorry you're smart.

Ben (10:45.428)
or I'm taking notes about something that we're talking about. I'll remember it that next day without having to look at the notes. I'll remember it two days later. But if I never use it within about a week, it's like my brain just does garbage collection. It's like, yeah, it wasn't that important. But if the fundamental basis of what you said, like some technique or something, like you're trying to teach me something about software engineering. And I'm like, that sounds important. I'm going to write that down real quick.

so that, cause I know that process of writing that down is going to help me remember the concept a bit more clearly. And then if we talk about it the next day, of course it's fresh in my mind. I'm going to remember it. And if we talk, like have that same conversation five days in a row, I'm probably going to remember all those details six months from now. It's going to be locked in there. It's all about like the recency and the frequency of information retrieval to make it kind of a permanent pathway.

Michael Berk (11:44.064)
Exactly. And theoretically, the human brain is optimized for this. There's algorithms that leverage spaced repetition to have you relearn things as you forget them. So it goes into long-term memory. And the brain is like a least recently used cache where stuff gets evicted when it's not being used. So that makes a ton of sense. but I just find it so interesting that both of us don't use note taking for retrieval. We use note taking for hardening and clarifying knowledge.

All right. Point one against the second brain. Another question, the second on our list is, all right, so if note taking at least for us is about clarifying and hardening knowledge, what do you go about learning? Is general knowledge valuable? Should you only learn these things that you know will apply to your day to day life? Should you learn because it's fun? Should you learn because you're curious? What are the guiding principles?

for knowledge being valuable.

Ben (12:48.46)
I'm about the worst human you probably know to ask that question to. And you could confirm by talking to my wife. I have an overabundance of completely useless information in my head. And that's an artifact of how I spent like winters in my childhood. I grew up in like Northeastern Connecticut, lots of snow days, lots of school cancellations. Instead of plopping down in front of

like the TV and just watching shows all day long or, you know, figuring out ways to just marginally not set the house on fire. I would just read books. and we had a huge library in the house that I grew up in. Not always the most scintillating, you know, material to read. wasn't like a ton of fiction or, or, you know, pulp novels or anything. Lots of reference books, lots of pretty good encyclopedia set.

I was just curious as a kid that curiosity never really went away. and I consume material, not just by written word, which is my preferred way, but also look, look at my YouTube video history and you'll be like, man, the algorithm must have a hard time with you. just random nerd stuff of like all these different topics that I

Michael Berk (14:09.486)
Thanks

Ben (14:16.93)
just what I'm curious about, I want to learn more about. And they're not high level videos. It's not like, this is the condensed 10 minute version of like a documentary. It's like, I get interested in stuff like there's this dude who is, I can't remember the name of his channel, but it was like a month ago, I got on a kick and I watched probably like 50 of his videos.

He'd go out in the middle of nowhere, buy this derelict car that's actually somewhat useful, but had been left like in a field for 40 years. And he just tried to fix it up in like an afternoon or within a couple of days and drive it back to his, his shop, which sometimes was like a thousand miles away and just seeing, and the guy's really funny. He's really clever. I think he's got like some show now, um, that, that spawn off of that, but it was just interesting day.

To see that, learn things about these classic cars. I'm not a motorhead, although I have in the past done stuff like rebuild engines with friends of mine, because I was just curious. They're like, hey man, my Trans Am engine blew up. What are you doing this weekend? I'm going to be right there and turning wrenches and figuring it out with you. thanks man. I'll bring the beer. Sure, dude. I don't know.

Michael Berk (15:32.078)
You

Ben (15:39.732)
an internal combustion engine works, but I'll figure it out. over a course of a couple of weekends, we rebuilt an engine. So I get a kick out of just like acquiring knowledge like that and acquiring skills more about like doing, but also watching stuff. It's entertaining to me because I'm just learning stuff.

Michael Berk (15:47.864)
That's crazy.

Michael Berk (15:58.67)
So you are purely curiosity driven. Like is there ever a filter where you're like, this is cool, but I will never need this under any scenario. I won't learn it.

Ben (16:08.472)
I used to say that about other things as well, like other topics. my go-to Zen channel on YouTube right now is Baumgartner Restoration. It's a dude in Chicago who restores jacked up old paintings. I have never taken an art class in my entire life. I have no idea how to paint. I'm not good at drawing, but

watching this dude's mechanical application of restoring paintings is fascinating to me. And just learning like, how do these things get created? Like, how did he develop these skills? And he's also kind of a nerd himself, so he builds like equipment for his studio. Yeah, I dig stuff like that. Even though I know nothing about art.

Michael Berk (16:59.662)
Got it. Okay, my take is I think a bit more first principle oriented, is like the purpose of knowledge from an action perspective is a bit different, but like knowledge acquisition should be fun and sustainable. And so you should just do what you like. Like we're on this earth for a hundred years or whatever, and it's really not.

worth it to try to reverse engineer action if it's really painful, if it's like pulling teeth. so that's the thing that I optimize for is just general enjoyment. and often that is curiosity, but also I find the abstract concepts to be like really just boring. I want it to be applied to something in front of me. So, I don't think I'll be watching those YouTube videos, although they sound great.

And instead I would be looking to learn through building. And that's always been my approach. uh, to give a very tangible example, um, I hate school. I like with a passion, um, no knock on people who like school, but I just never understood the purpose. And so in junior year of high school for my capstone project for English class, we were supposed to write a paper, right? And instead I wrote an objective C app.

that would quiz you on one of the books via simple game mechanics. And I printed out all of the code as my paper. And I got an A because clearly I showed a lot of initiative. But that's where the English teacher was like, who the fuck is this kid? Why is he here? And so it's those types of things where if I can actually put it to a project, I really enjoy it.

Ben (18:35.672)
Thanks

Michael Berk (18:54.082)
but general knowledge acquisition for me is really challenging. So I don't usually do that. It needs to be at least one link adjacent to what I'm generally into at the time. But I have a question for you. You have these trillions of facts just swarming around in your brain and your recall is really good. Like that's objectively one of your skills. and also seeing how they interleave and like how they they're related. How often does that miscellaneous knowledge help you?

in a practical sense, like solving a bug.

Ben (19:28.246)
Yeah, that's the funny thing.

I've been on this planet for a while and I've had a lot of temporary interests in things. Some of it has stuck around for quite a while, but a good tangible example for this is

I remember back in, must have been like 2005, I became really fascinated with like optics, particularly in like cameras. like old SLR, like single lens film cameras. had a number of them. I used to go and shoot photography. I was like really fascinated by it as just, you know, prior to the digital age and

I got really interested in how film actually worked, the mechanics of it and the chemistry of it. And learned just enough to realize how much of an ignoramus I was about it. And I was like, wow, this is really complicated. I don't really want to go that deep into understanding this stuff, but I do enjoy the whole concept of, I can get these different effects if I under overexpose and I can do double exposures and how that changes.

like saturation of colors. So I ended up learning a lot about color space theory with like, with regards to photography about like what makes sense in a framing of a photograph of like what colors would would look nice together. Like how should I frame this? I used to do like a lot of landscape photography. So like, where should I actually frame this in order to get the best shot?

Ben (21:16.728)
Or how should I do like an extended exposure and put filters on the front in order to limit the amount of light coming in? Fast forward 14 years after that, and I'm stuck at a problem at work where I have to resolve color space. And if starting from ground zero of like, oh, you're a data scientist and you have to deal with these images, I would have had to like,

just start researching stuff and like trying to learn like, how do you do color manipulations? go check this library that you can use and see what APIs are available. And there are a couple of people on the team and we were all kind of trying to tackle this problem. And we ended up going with this really weird thing that I did, which was converting the digital data, the RGB data into HSV space. Cause that's like how

the human eye perceives color and how you can kind of trick an algorithm in order to resolve color spaces that the human eye perceives as being dominant or as being important in an image when you're putting colors next to one another. And it worked out, you know, the complexity reduction in the implementation was very significant compared to what we were originally

planning to do because of all this stupid knowledge that I had acquired years before. And I had bought books and stuff on the topic and I just went back, like went home. I was like, I'm going to read about some of this stuff and just flipping through. was like, I remember that. remember that. Like, bet that like, this seems just like very basic trigonometry to solve for some of this stuff. What if I just whipped up some functions? I don't think I need this big package. And yeah, it ended up working.

I can't even count how many times that's happened in my career. Just like some random thing that I was interested in just becomes all of sudden imminently relevant to what I'm like a problem I really need to solve.

Michael Berk (23:28.408)
Two sub questions. First one, would you have been able to come to the same solution if you had not had that curiosity driven exploration in the past?

Ben (23:41.272)
Probably not. Nobody else on the team was thinking in that way. If we got an expert in image theory on the team, or if found one, we probably would have found that solution even faster than having me to think through that problem. But with the team that we had, yeah, nobody was really thinking in that way.

Michael Berk (24:04.302)
negates my second question, but let's say the answer was yes. Let's say after some amount of hours you guys would have come up with this HSV solution. Do you think you would have spent less or more time

frame this. Summing up all of the curiosity driven exploration, those total number of hours versus the applicable problems that though that that information informs. Is it more efficient to just have done your broad curiosity driven exploration and then have them happen to land into N number of problems and solve those better? Or is it more efficient to case by case for each problem, learn all the context just in time?

Ben (24:51.32)
That's an excellent question. I don't think about efficiency like that. That would stress me out, man. Like really stress me out. I know people who are like that though. I work with people who are like that. That they go super deep on whatever problem is immediately right in front of them. But even those people without, if that's your mantra, like how you're going through life.

Michael Berk (24:57.047)
Me neither.

Ben (25:17.246)
It in the moment, it seems like that's just what that person is focusing on. It's just, I need to immediately learn this information right now. But if you take that same person five years later, it's going to be the same sort of effect. That's like, they learned all that stuff. They're exposed to all that stuff, which is going to inform intuition later on.

Ben (25:42.05)
So it'll apply in other ways later on, the same way as if you're just randomly going about whatever you have curiosity about.

Michael Berk (25:53.55)
Yeah, it's in-

Ben (25:53.592)
that might be more applicable within a particular industry. Like if you're a software engineer and you're just going super deep on certain problems, you're going to have this intuitive context that is probably unmatchable amongst your peers because you're just going super deep and really understanding these concepts in the moment versus somebody who's just broadly, you know, exposing themselves to the

the entire tableau of human existence.

Michael Berk (26:25.634)
Yeah. Yeah. Cause that's the problem statement of this episode, right? Is knowledge acquisition for action, for value, and not just for fun. So that's the calculus, right? It's like, if time is your unit, is it more efficient to learn just in time or have curiosity driven general knowledge and then have it come in when you need it? So it's interesting. that segues into another question, which I think is super relevant, which is

Ben (26:35.128)
Mm-hmm.

Michael Berk (26:57.826)
This, this whole set digital second brain thing, it doesn't click for me super hard because it's, find it challenging to leverage facts. instead I find information really valuable when it becomes intuitive and subconscious and you're then able to leverage it like a tool almost. It's not a discrete piece of information. It has a framework. It has a feel. has a, I don't know, a trajectory and identity. And so for that.

aspect, I think one of my strengths as like just a human brain is I'm good at getting stuff to become intuitive, and I'm usually faster than most people at it. And I'm not quite sure why I think it's being first principle oriented, but I'm curious, in your experience, when you're looking to acquire knowledge, and you have the sort of a discrete piece of facts, when does it become intuition so that you can leverage the sort of the principles of that information on other problems?

Ben (28:01.936)
actually, I don't actually, I don't think through problems in that way. So I always go with like an internal monologue self-evaluation of anything that I'm presented with to try to map out contextual clues that lead me to think that I know what is being discussed or presented to me. I think that's like a normal human reaction.

Michael Berk (28:09.518)
Hmm.

Ben (28:29.816)
feel like, hey, there's this novel problem that we're discussing. My brain is going into overdrive trying to create a pattern of like, do I know what this is? Or is there a corollary in my experience that allows me to understand this abstract concept? If there is, then I start attempting to map that to my own experience. Like some technical problem that's coming up, like, have I dealt with something?

Michael Berk (28:58.606)
Can you give a simple example, like a walkthrough?

Ben (29:01.833)
simple example.

Ben (29:10.806)
I mean, I was in a discussion yesterday where people are, we were sitting down and discussing my intro today to this episode about safe code execution and like, what are the ways that that could possibly be needed or why would that be needed? And there were a couple of discussions going back and forth with people of like, would a user write a function that could do X or

Do we even need safe execution because of this other condition that somebody might write? And I started to think back to some stupid stuff I've done to troll people in previous jobs that I've had. You have a group of people that all work pretty well together. You kind of clown around and there's just

this PC sitting in the corner in a room that's slated for recycling. know, like somebody has the great idea of like, Hey, that thing's going to be taken out of here. And like tomorrow, do you think we could like, like seriously mess that thing up? It's going to be thrown away. Anyway, nobody's ever going to turn it on again. I'm like, hope I beer and just seeing like, what could I do to this?

Michael Berk (30:35.598)
You

Ben (30:39.796)
machine, like what could I do with the command prompt and some good old bash scripting, or I could wreck this PC and just getting creative with it. And it let me fundamentally understand the concept of like big O notation. Like what is computationally complex? What sorts of crazy things could you accidentally do even with just a simple typo if you're not really careful about what you're doing?

And then showing everybody like, check it out. I can fill the hard drive in this thing to the point where the file system indexer won't even work anymore. People are like, what? There's no way. Like, no, no, for real. I'm going to create a couple hundred billion, you know, one byte files and just break windows. They're like, okay, show me. Write a script, do it. And then sure enough.

search function doesn't work in Windows File Explorer. They're like, what the heck? I'm like, I filled up the tree. They're like, well, how did you even do that? You can check the script. But yeah, we basically only have a prompt now because Windows won't start.

You know, it's interesting things like that. We're just exploring and creatively trying to do something that was, you know, destructive. But that informed that discussion yesterday where I was like, well, okay, let me take screen control. I'm going to show you guys something. And wrote a two line function that I asked the people on the call. like, what do you think is going to happen? And they're like, well, it's.

It's, it's going to run. It'll just take a while to run. no, no, no. Like if we run this, the machine's not going to respond. It's a virtual machine. It's a pod and Kubernetes, but I'm going to kill that pod by running this. We won't be able to stop it. It'll just keep on running until we physically like, you know, P kill that machine. And people are like, no, no, that's not how it works. And I was like, all right, execute seven minutes later, still running.

Ben (32:54.494)
all abort commands are not responding. was like, now do you believe me? This is how dangerous this is. And then I'm like, how do you even know how to do that? And like, that's a long story. But yeah.

Completely irrelevant skill set to have of just knowing how to do stupid things on a computer actually allowed me to convey to people like how important it is to have this functionality.

Michael Berk (33:22.252)
to the original question, when did it become intuition? It seems like it became intuition when you actually got hands on keyboard and tried stuff, which is my angle. Like, can you read a book and have it become like, I guess I can as well, but I think that playing with the information to solve something or to create something is the best way to make it become intuitive.

Ben (33:46.968)
For sure. I couldn't even tell you how many instances I saw. This will be going in the way back machine right now in the United States Navy. We had some people at the nuclear reactor prototype that I taught at that were probably, know, IQ is well above 150. Like some very highly intelligent people. They could regurgitate loads of theory about

how the electrical distribution system works on a, you know, United States nuclear powered submarine. They knew how the steam system worked in the secondary plant. They can name off all the valves in the steam system, you know, all 900 of them. And they understood fundamentally why the system is designed the way it is and how it's built. And then you get these people who were, you know, straight A's.

in the academic courses and at the prototype you have to do on the job training. So there's a qualified instructor that is watching what you're doing and making sure that you can turn those valves and that you can operate that electrical power plant and you know how to physically do this stuff. And we would have every class up 300 or so students that would come in and they're there for six months.

to get this training and qualifications. And there's about 50 that were top academic performers that just failed out because they could not apply any of that. And I would ask some of these people and be like, did you ever build anything as a kid? Did you ever like work with your hands to understand how mechanical things work? Did you ever disassemble a telephone and put it back together? Did you ever do anything where you like

had to physically figure out how something is built and marry in your mind the theory to that. And every single one of them, of these students that ended up failing out because of physical performance stuff, they all said, no, I never did stuff like that. I just studied. There's a huge disconnect in human ability if you never apply the knowledge that you gain.

Michael Berk (36:07.79)
Cool. So that's a really interesting angle that like confirms my approach, which is to make knowledge intuitive, you have to use it and you have to like play with it and apply it. It's find its limitations, find its strengths, find its weaknesses. But I think where we differ is that I leverage that to be the source of my knowledge, like the top of the funnel, whereas you leverage curiosity. So you're like, ah, this is cool.

But I sort of reverse engineer, and I only learn stuff just in time, typically. But I'm also trying to change that, because I think there are weaknesses. Like you mentioned, the HSV example, general knowledge comes up in really weird and valuable ways. So OK, cool. We have butchered the order of our questions per usual. Ben, you want to do the next one?

Ben (37:02.23)
Yeah, do you feel that the knowledge that you've gained outside of tech, like hobbies, things that you've done in the past, and the skills that you've gained by actually applying those things?

Do you think that that helps, like the broadness of being exposed to many different topics helps the way that you actually process how to evaluate a new novel problem?

Michael Berk (37:30.712)
to evaluate a new novel problem.

Michael Berk (37:38.902)
I think answering it from like a step back indirectly, I think that the way that I treat my brain is sort of like a black box, like let's call it a neural network. And there is logic there's, which is sort of the more analytical sort of tops part. Then you go down to more subconscious and then all the way down, maybe to unconscious animalistic fight or flight, those types of responses. And it's sort of a continuum beyond that. I don't know how the hell it works.

All I know is that when you put in an input, stuff happens and then there's an output. And so I've sort of taken that approach and tried to just listen to see what inputs provide and what output comes out. And I've noticed that

Michael Berk (38:27.534)
Getting knowledge to be deep, like to be sort of in the subconscious, not on the unconscious level, that's like hard to mess with, but in the unconscious level, that's so freaking valuable because you can apply philosophies to different novel problems and to the evaluation component of the problem as well. So really simple example, hyperparameter tuning. That was like really interesting to me and applicable to one of my problems for, I don't know, about like a six month project. And I really studied it.

not like at a scientific level, but I've read papers. I wrote a blog that deconstructs Bayesian optimization versus all these other methods. And that weirdly has been one of the most truly impactful philosophies I've ever learned because so much of life is searching through an unmapped space. So you have local information. You need to go somewhere else that you don't know how to get there. What's the most efficient way to search through

Ben (39:18.349)
Mm-hmm.

Michael Berk (39:26.222)
solutions to search through, I don't know, just like even paths in your life. Uh, like let's say I want to be, uh, the founder of a really impactful ocean startup, which is true. How the hell do I get there in like a five, 10 year timeline? I don't know, but I do have the local information. I can leverage the cassock gradient descent to take the like highest derivative step.

I can go sample a bunch of points along the path and then reverse engineer what the curve might look like. So there's all these really cool applications that, as long as it gets deep enough to become philosophy and intuitive, it's really, really valuable. And so with all that, I think the brain is not just professional, but it's life and whatever you feed into it. It all reinforces itself. So like going to the gym helps me be social, helps me with work, et cetera.

It's all one thing. And I think that limiting it to work and technical things is very short-sighted.

Ben (40:32.642)
That brings up an excellent question as well that I think is even more important to ask. How important is it if you're working in tech to hit the gym?

Michael Berk (40:44.206)
I think it's important for literally every human, no matter what you do. But, the key thing is like thinking about this holistically, right? Like your brain is one brain. When you're in work mode, it's still the same brain. When you're in social mode, it's still the same brain. When you're sleeping, it's still the same brain. And so what does the brain want? It wants balance and variety. And for me, it wants extremes. So I tend to work very hard and then I tend to fully.

like disconnect and go work out very hard or be social, very hard or whatever it might be. And that balance allows me to be more extreme in each of these scenarios. So working out specifically isn't like essential. I then usually like six times a week, with like yoga, basketball weights, try to run even though running sucks. and yeah, it's like, I would be a mess if I didn't have a rigorous workout routine, but it's because

I go so hard in all the other areas that the balance is needed and finding like what exactly constitutes balance. Like what are the different areas that you need to like quote unquote go hard in is somewhat challenging and like trial and error. But generally those are the things that I think about and I want to give my brain variety, but what's your take.

Ben (42:05.656)
mean, speaking from personal experience of working in tech and not working out, is years ago when I was kind of your age, I was spending pretty much all of my free time doing outdoor activities. not really, I mean, early on when I was in the Navy, was like hitting the gym, I was like, oh, I'm super weak. I want to get strong. And

Michael Berk (42:25.39)
Hmm.

Ben (42:35.212)
did that for a couple of years and then got really interested in other forms of extreme physical activity. Rock climbing was big and I also did a lot of backpacking and not the sort of camping that most people probably do. was more like get a pack, 50, 60 pound pack full of enough food for a week or a week and a half, a bivy sack, no tent, a bunch of stuff that were

essential survival gear, not pack a bunch of junk and just go to a state forest and get lost for a week and just camp in the middle of the woods or camp by a lake or something. And every day walk 15, 16 miles and then set up camp again. And yeah, and then doing like the rock climbing stuff to the point where,

so much gym rock climbing never really got into that but the outdoor stuff like I had thousands of dollars of gear that I had like many ropes I had like a whole room in an apartment I was living in it was just like the climbing gear room so you name it I had it like all the gear and I would go do lead climbing with with friends of mine so lead climbing is for those who don't know you're the person who's got all the the camelots and

You pitons and stuff. You're, you're you're carrying all the rope around you or on your back and you're climbing up a rock wall that has no connection points for a rope and you're putting in those connection points as you go. And, you fall a lot if you're, know, at first. And hopefully you put your gear in well enough that you're not going to, you know, be convalescing for a month before you can go out again. But, yeah, it was like super into that type of stuff. And then.

Michael Berk (44:30.542)
Convalescing.

Ben (44:33.612)
got into tech later on in life and realized that I didn't know as much as I thought. So I was like, I got to work extra hard and I got to get like all this work that I'm, you know, putting on my own shoulders and trying to figure stuff out. And I became pretty sedentary and noticed that it was like, man, I just don't think as clearly. Maybe I'm just getting older, losing touch. And

It was a couple of years ago, my wife was like, hey, she's at the gym. Like we have a garage, why don't we just build gym down there? And I was like, you know what, you're right. Like that day ordered a bunch of equipment and stuff. And I've been doing every other day, you know, hard training in our gym. And it's amazing how much more mental focus I have now. It allows me to sleep better because I've got issues with.

And I always have since a kid, like my brain doesn't shut off properly. So if I, if I'm in sedentary mode and not like really punishing my body with lifting super heavy shit, I've just won't get tired enough. I'll be up till like three 30 in the morning. It's really unhealthy, but I can't actually get to sleep. So yeah, it's great. Particularly for people in tech. can't recommend it enough.

Michael Berk (45:58.668)
Yeah, not fully agree.

Ben (46:01.346)
Plus it feels good where you're like, Hey, I can lift like a lot of weight and it's nice to feel strong.

Michael Berk (46:08.908)
Yeah, yeah, I feel like humans naturally like to be good at things. So I don't know. But I completely agree. All right, in the interest of time, I will ask one more question. All right, this is an AI podcast in theory. So we're going to talk about tech. In your opinion, is it more important to...

be nature skilled or nurture skilled.

Ben (46:43.874)
nature or nurture in tech?

Michael Berk (46:49.644)
And you know what? I'll read the sub questions. Can you take a Muppet and create a competent engineer? Can you take a highly intelligent person and create a shite engineer?

Ben (47:00.758)
Yeah, I added those two questions to this one. Can you take an absolute muppet and make them into somebody who's competent? Yes, it is possible. I've done it.

Michael Berk (47:14.434)
Have you seen it?

Got it.

Ben (47:18.584)
I've taken somebody who was super well intentioned, who is like really hardworking, passionate, and had no clue what they were doing and made them, got them to the point where they were building on average, like two fairly medium sized to large size data science projects per month to the point where like the first time that I interacted with the person.

It was somebody who had worked at the place when I got brought in to kind of lead a data science team. And they were in this dual hat role of like analysts slash, you know, air quote data scientists just had no clue what they were doing. they could use API is kind of, but they didn't have any foundation of like how you have to apply like the scientific method to what you're doing.

what's so important about certain stages of model development and how to do stuff like test whether your hypothesis is correct and evaluate the data that you're generating out of the artifacts of this model. within a year and a half, yet it was a rock star. And it's totally possible. It's not easy and

You have to present, you have to figure out how that person learns best and then give them a graduation of complexity over time and push them into spaces that you know they're going to flounder and fail at, but be there to catch them when they do and then set them on the right track. So it takes a lot of effort and time, but every time that I've done that and it's been successful has been very fulfilling.

It's like, hey, I just set this person up for potentially a career.

Michael Berk (49:15.18)
Mm-hmm. Yeah.

Ben (49:16.504)
where they may have gotten there on their own eventually, I don't know. But I could have potentially just short-circuited years of struggle and effort for this person.

Michael Berk (49:29.312)
What I think though that you couldn't have done that if they didn't have growth mindset and like a few other nature qualities. What do you think are those nature qualities or can you truly turn an absolute Muppet into a good engineer?

Ben (49:43.404)
Now, like that core person has to be humble. They have to be willing to learn. They can't be like too hard on themselves and they have to, they had definitely, as you said, they have to have that growth mindset. They have to want to get better at something. And this, this guy was very passionate. Just didn't have any of the skills or knowledge to do what he wanted to do.

Ben (50:13.686)
And it wasn't like...

Michael Berk (50:13.763)
Got it.

Ben (50:18.882)
Like with a sufficiently complex enough profession, when you're looking at it, if you're just left to your own devices, like, yeah, some complete asshole could be like, well, there's the internet, all the information's out there. True. But if you've ever tried to go from, from zero to one on something like data science, if you don't have any background in it, it is just this insurmountable wall of complexity.

Michael Berk (50:47.468)
Yeah, it's not trivial. Yeah.

Ben (50:48.522)
Like, how do you get started in this if you don't have somebody to be like, Hey, let's walk through this. And there's a huge difference between like, when I saw some of the work that he had done originally, where he was like showing me things, he's like, yeah, I kind of based this off of this Kaggle example that I saw. I'm sitting there like sitting down with him on like the first week of, you know, working with him.

Michael Berk (50:56.525)
Yeah.

Ben (51:15.864)
I was like, let's just walk through the code, like line by line, what you have written here. let's like, why are we doing this? What is important here? Why is this not working the way that we want on the real data of the company? And it was a way of jumpstarting his understanding of why things are the way they are.

Michael Berk (51:37.528)
Yeah.

Noted. Okay, cool. I...

Ben (51:42.614)
And can't the second part, can you take a highly intelligent person and create a complete shit engineer? Yeah, you can definitely do that by be demeaning to them and giving them the worst projects and just having unreasonable expectations.

Michael Berk (51:46.946)
Well, that goes without saying.

Have fun.

Mm-hmm.

Michael Berk (52:01.112)
Yeah. Yeah. Humans are humans. Like you can break a human. Yeah. A hundred percent. Okay. Final question that I want to chat about, cause I think it's an interesting philosophical jump and a nice ending point. We've talked a lot about how the purpose of note taking is for knowledge hardening and clarifying thoughts. For retrieval, I am personally under utilizing my notes to some degree.

it sounds like you might be as well. Maybe retrieval isn't super valuable for you, or it's just in your head because you've already hardened it. But in the age of AI, theoretically, you can leverage the second brain to be a lot more, relevant and a lot more concise and maybe even almost self-organizing. So there's discrete pieces of knowledge that can find associations, find relevance to whatever you're doing in the current time. There's a lot that can be done.

Ben (52:40.984)
Mm-hmm.

Michael Berk (53:00.856)
What are your thoughts on this?

Ben (53:05.676)
I can tell you what my bias for action was on this exact topic. I was really curious if I could get OpenAI to simulate me. I do have the ability to...

Michael Berk (53:09.325)
Yeah, hit me.

Ben (53:26.37)
kind of have a data set that's out there that's been published that is in the tone of how I communicate, not when speaking, but when writing. So I did build an agent, several months back. I was like, all right, the data that you can retrieve and answer questions about machine learning is the book that I wrote. And I had that configured with a

a suite of tools, just locally executing functions for it to, you know, test some things out. Like, Hey, if you have code, you're going to generate, run it here. And then, you know, that's what this tool is going to be for. And then the retrieval is just putting it into a vector database, the actual corpus of that book. And it was a little weird at first. Like it felt like I was actually talking to myself. It was, it was kind of weird, but I had started asking it.

questions about things that I knew that were topics in the book and what its thoughts were on it. And it was definitely creating new content and extrapolating information from the text and providing references to it, but expanding upon some of the topics that I was asking. was like, this is kind of awesome. I'm going to see what else I can do with something like this.

and started loading up other, you know, texts that I owned copyright to because I purchased the book in electronic form and then started loading it up in the vector database and started asking all of these different questions about, you know, software engineering things about kind of theory. And I was pleasantly surprised at like how, how great the interaction was.

Michael Berk (55:19.936)
Interesting.

Ben (55:19.97)
So it's entirely possible in today's tech to do stuff like this. If you want a selective corpus of information that you trust or that you're just messing around with, they're pretty powerful.

Michael Berk (55:32.803)
Hmm.

Cool. I have two thoughts that is bad-ass though. and I'll have more thoughts later, I'm sure. But, two sort of unrelated thoughts. The first is there's been a trend over history that D values knowledge holding and values what you can do with knowledge. So back in like the 1900s, the doctors were the doctors because they knew the stuff.

Then like there were books that you could like theoretically go get knowledge from public libraries. The value of a fact became less valuable than with the internet. became way less valuable. It's more about sorting facts to figure out what is the true fact and leveraging that information. I feel like now with AI it's even more so that the true value isn't rote memorization. It's the ability to leverage information, recombine it, like smush it around to something useful. So.

That's just like one piece of like trajectory that I think would inform how people would think about this retrieval component specifically for your own personal knowledge. And with that, I personally try to invest in my brain over just knowing things. I try to like be dynamic and intuitive. and the second piece is, I used to work with this nonprofit and now I'm on the board with them. and with the founder of it and I were chatting and he was like,

teaching himself like stat one-on-one and he's an engineering, software engineer and a manager. And I was like, why the hell are you doing that? And he was like, well, I think general knowledge is really valuable. Like that whole discussion we were talking about earlier. And I've sort of opted for the opposite, which is to be a just in time compiler with the maximum amount of RAM. So I don't need all these like facts stored as long as I can go into like medium disk SSD type memory and retrieve them just in time.

Michael Berk (57:30.792)
but I think that's kind of wrong and I'm missing something. So I'm iterating on it, but I'm hoping that stuff like rag will facilitate the two disc retrieval so that my brain can be even more free to not remember facts. But I need to know how to like get home and like, like speak English. But beyond that, I love not knowing stuff. It's so like the brain can just go more. feel like less constrained.

Ben (57:57.688)
Yeah, I mean, my hot take on where Gen.ai is right now and where it's moving in the future is like when OpenAI released, you know, GPT-2 and we're like, hey, this thing is actually capable of like managing conversations. It hallucinated like crazy all the time and it wasn't particularly useful for like work.

until GPT-3 came and you're like, okay, this is a little bit better. people see this almost as like this, an entity. I've heard people throw pronouns at it before. it's like our habit of seeing something like that as like the way our brains work is we almost think that it's just another person that we're conversing with.

But where we are right now with like 03 Mini-Hi, right? Which is pretty advanced model from opening high. It's fantastic. I see it not as, I see it basically as a mirror of society and how society has evolved over hundreds of thousands of years as humans, where we've gone from generalism, just like, hey, go back like,

few thousand years ago, everybody needed to know how to farm. You had to get your own food. Everybody needed to know how to do basic building. You have to build shelter to the point where in the modern era, everything's so specialized to the point where it's like the processes involved in anything that we do as a species are so complex that you have to specialize in something and there has to be somebody who's really good at this thing or a group of people that are really good at this thing.

You know, nobody's going to be building a football stadium with just general knowledge. You know, there's a lot of experts involved in that. Everybody from, you know, the actual people on the ground operating cranes and moving things into position and, you know, figuring out where do things need to go to the people who are doing metallurgy back in, you know, the foundries that are making the steel.

Michael Berk (01:00:02.424)
That's a funny concept.

Ben (01:00:23.084)
people that are involved in mining of that material to get to the foundry. There's specialization in all of this stuff. So I see Gen.ai and advanced LLMs in general and these mixture of expert models that are proliferating. Those as being corollaries to the advancement of our own journey as a human species. These systems are evolving as well.

So the original GPT-2 is just like, it's a kind of smart person kind of GPT-4 is like, yeah, this is somebody with a lot of knowledge. You start building these agentic systems where there's specialists involved there. And we're now mirroring that where we're getting these experts that are really good, like superhuman qualities. So we're almost seeing it as like a society within a society of like, here's these experts. And that's where I think it's going. And.

Michael Berk (01:01:09.454)
That's badass.

Michael Berk (01:01:14.478)
What the hell?

Ben (01:01:19.158)
the tooling that we're building right now is giving.

Michael Berk (01:01:21.09)
Are they going to build governments and schools?

Ben (01:01:23.896)
I mean, there will be government regulation of this stuff someday, giving additional technologies and tools to this society of knowledge and reasoning and making it so that these entities can verify the veracity of their claims is what's going to

Michael Berk (01:01:28.632)
Fair.

Ben (01:01:52.994)
propel us in a different direction as a species.

Ben (01:01:58.87)
I mean, we are on the precipice. I think life is going to be quite a bit different in the next 10 years.

not just with, I'm embedding a chat agent in my app, or I'm doing this interesting thing with, you know.

basically data cleanup by using GNI, it's going to advance to the point where we're having this embedded as assistance in helping our own society struggle, like resolve truly complex problems where you need a whole bunch of experts working together and having a shared context, which is just not really possible in human society right now. We are too big. We're too disconnected.

Michael Berk (01:02:52.482)
Wow, that's a trip.

Ben (01:02:57.24)
But like, what do you think would happen if we were able to get a performant, highly, highly capable LLM that actually had direct access to all knowledge, all like verified, correct knowledge. If you hooked up an agent to just a crap load of vector databases that stored like all of the low level details about something like let's say metallurgy. Like, Hey,

every patent that's ever been issued, every research paper that's ever been done about steel manufacturing, and then started asking questions or just feeding in data of results of tests that you've been doing about like, hey, I want to, I'm looking to like make a stronger steel that's more economically efficient to produce and less environmentally impactful. What are some ways that we could think through this? And then it just goes off for an hour.

collecting all of this data, burning through millions of GPU hours and presents this report of distillation of all this knowledge. Potentially finding patterns that we didn't find, or it's just too voluminous of data for humans to actually understand.

Michael Berk (01:04:19.854)
Cool, yeah, let's in the last 30 seconds of this episode answer that. That's another episode. But that's a really interesting angle. I have some thoughts, I think. But it does make sense that they would become building blocks. Like most things become building blocks, like most pieces of technology. So.

Ben (01:04:40.044)
Mm-hmm.

Michael Berk (01:04:44.79)
Yeah. Wow. Okay. Anyway, I'll summarize. Cool. So this episode, talked about knowledge acquisition for action. So not just for the sake of gathering knowledge, but actually using this knowledge so that you can be better at stuff. And what Ben and I sort of generally concluded is that for us, at least the purpose of knowledge and notes specifically is to harden.

the knowledge acquisition process. It's not for recall. It's not going up and looking up things. but I know a lot of people that disagree when you're going about acquiring knowledge, make sure you're sustainable. So whatever drives you leverage that for me, it's building for Ben, it's curiosity for you might be something different. and then general knowledge seems to be really useful for these zero to one ideas that you couldn't iterate towards. So that's the sort of the application of broad knowledge is so that you can come up with.

completely novel solutions instead of just making a thing incrementally better. Use knowledge to do stuff. That's how you make it intuition. And then if you're looking to better yourself and go from a not so great engineer to a much better one, the nature requirements, at least according to Ben, needs to be humble, hardworking, and then willing to learn and have a growth mindset. And then finally, for the last high level philosophical point,

Seems like AI is starting to reflect society a bit. So with this knowledge acquisition discussion, might start doing some, or it might start doing some of the, might start be doing, wow. My mind just broke real quick. It might start doing some of the actual thinking for you, not just the recall. And so figuring out how to leverage that as a building block. So you're sort of an orchestrator of these thinkers, not just these recallers.

That could be a really interesting thought experiment. So despite the stutter at the end, anything else?

Michael Berk (01:06:46.678)
Well, until next time, it's been Michael Berk, and my co-host. And have a good day, everyone.

Ben (01:06:50.776)
We will catch you next time.