Tom Babb: Byron, you can probably just introduce yourself and… Kill a couple minutes here.
Bryon Mackay: Oh, sure. Okay.
Bryon Mackay: Alright!
Bryon Mackay: Alright, well, welcome, everybody.
Bryon Mackay: People are coming in. Appreciate y'all coming, making time in your…
Bryon Mackay: afternoons and evenings to come be here. My name is Byron. I'm gonna be your speaker tonight, and I am the Director of Learning here at Gauntlet, so I really focus a lot on the curriculum, making sure we're keeping things up to date.
Bryon Mackay: Making sure you have the most pertinent information when you go out, and you have your job, and you're an AI engineer or an AI-first engineer. So, it's an interesting problem to solve, honestly. As many of you can probably attest, keeping up with AI is really hard.
Bryon Mackay: There is a lot going on.
Bryon Mackay: And, you know, the… this… this topic we're talking about tonight has really started to…
Bryon Mackay: surface its ugly head in a way that we all kind of knew it was happening, and then it had just really, like, hit the wall hard. Where we start seeing, like, major issues with major companies on their apps, and everyone's trying to figure out what to do about this. Now.
Bryon Mackay: You may not have felt this as much, maybe in your roles, or maybe in the apps that you've used, but I'm gonna show some metrics here in a second here that are…
Bryon Mackay: hopefully gonna be a bit more eye-opening. It was definitely for me. And then we're gonna talk about how to, how to, you know, stop this from happening. It's interesting, though, I will say that…
Bryon Mackay: There are a number of apps that I use. I won't name names, but they're all big companies, I'm sure you use a lot of the same apps, where I have just found the most random and annoying bugs that I have never seen before.
Bryon Mackay: And I don't know, I don't think it's a configuration thing, but some of the most simplest of actions will cause something to break.
Bryon Mackay: And I'm just… I just sit here, I sit back in my chair, and I go, this is an AI problem. This is a uniquely AI problem where we are seeing
Bryon Mackay: So many more issues. But then the other shocker is these issues don't get resolved, right? They just kind of stick around for a while. So anyway, there's all sorts of big problems and little problems that we're seeing.
Bryon Mackay: But our jobs as engineers is to fix it, yeah.
Tom Babb: Quick question, sorry, we didn't address this, but how do you want questions? Some people, they're able to raise their hands. Do you want them to be able to ask questions along the way, or would you prefer that all questions are safe to the end?
Bryon Mackay: You know what? Let's have him do it on the way. I'll find some spots to pause.
Bryon Mackay: And then we can take some questions. I think that's tough. Yeah.
Tom Babb: I'll say to the audience real quick, if you have a question, there's a little button that says, raise your hand, and that will put you in queue, and then when Byron's ready, we'll click allow to talk, and you can ask your question, and we'll just go through questions that way.
Bryon Mackay: Awesome.
Bryon Mackay: Okay.
Bryon Mackay: Austin, we will not be covering that. We will not be covering how to bundle map files in our automated updates. That's not quite the scope of this one, it's more of a detail, but, you know, we'll, we'll get there. So, alright.
Bryon Mackay: Let's go ahead and get started. Oh, let me give you one more bit of an intro here. So, Tom reminded me of this just a second ago, and I… it's an analogy that I love, and that I,
Bryon Mackay: It's a little bit of a personal spot for me, so just so you all know, I spent 10 years as a mobile engineer, that's all I did. I did Android and I did iOS. And at the time, that was really rare, to do both.
Bryon Mackay: I do have my preferences for one versus the other, but I did a lot of work in each, and I saw the mobile revolution happen as a developer, and so when the App Store came out, like.
Bryon Mackay: we were all just going crazy trying to build these things, and… but the interesting thing is, what we're seeing today with AI is very similar to what we saw with the mobile revolution. And what I mean by that is, when apps first came out.
Bryon Mackay: When apps first… when the app store opened up and everybody was building their apps, they all looked like mini web pages.
Bryon Mackay: And so we all got these apps that looked like mini-web pages, because that's what we knew.
Bryon Mackay: And all the apps that we had in the apps were similar to that, were like web pages. And then things started happening. I remember specific libraries coming out.
Bryon Mackay: I remember when Facebook introduced the hamburger menu, and that was just, like, blew everybody's mind. We're like, what a hamburger menu? You gotta be kidding me! That's amazing! Apple had their human interface guidelines that really steered the way that, that… and still influences how mobile apps are today.
Bryon Mackay: But over those years, then we started to identify there are applications that we can do that we've never done before because of a smartphone.
Bryon Mackay: And so, for example, there was no web app to call a random stranger to come pick you up.
Bryon Mackay: That didn't exist.
Bryon Mackay: And it couldn't exist, because we didn't have a good way of doing it, and now we do, with phones.
Bryon Mackay: We are at… the early mobile stage of AI right now.
Bryon Mackay: There are a lot of things we are still trying to figure out, every single day, about how to use this, how to build with this, and I… it's my opinion that we are not thinking big enough.
Bryon Mackay: That there is… at some point, someone's gonna hit it, and it's gonna nail a concept, and suddenly it's gonna just absolutely explode all over again. Or we're gonna realize the great power and capabilities that we have with these applications. And the application will probably change.
Bryon Mackay: The way that we see them, the way that we use them, and right now, we're just using the paradigms that we know. We know websites, we know chatbots, right?
Bryon Mackay: That's the current paradigm. But what if it wasn't chatbots? What if we figure out other ways to do it that are not… that don't look like chatbots?
Bryon Mackay: I don't know the answers, I don't think anybody does quite yet, but I do feel like we're in those early stages, just like we were in mobile, where we're transitioning from what we can do right now
Bryon Mackay: To what we can actually really do once we really harness this and think about it differently.
Bryon Mackay: Part of this conversation today is going to be about that, about some of the lessons we're learning, and the hard lessons we're learning, so let's get into it.
Bryon Mackay: Alright, who here is vibe-coded? I don't know if y'all have access to emojis, but I'm gonna assume all of you are raising your hand. We've all vibe-coded, right? It's a fun time. Wow, it makes programming so fun and so fast. You can just, with literally anything you can dream, you can see it.
Bryon Mackay: And it's amazing.
Bryon Mackay: So, there's… there's problems with that, right? Right? I… I have Vibecoded an app with my son, and I'm afraid to look at the code, right? I… I don't even want to know what's under… what's underneath that blanket. So we'll…
Bryon Mackay: But anyway, so how do we fix this? So we can't do this in our jobs. We're engineers. We've got to make sure things are quality, that things are good.
Bryon Mackay: Here's some, interesting stats that I think will, highlight just how prevalent these issues are that we're seeing. Ai-generated PRs contain 1.7 times more issues overall than if a human did it.
Bryon Mackay: So, we're getting more issues. Are they sneaking through? You bet they are.
Bryon Mackay: So, this isn't a reason to stop AI, this is a reason for us to stop and consider what we're doing with AI.
Bryon Mackay: You know, I thought about this the other… last night as I was preparing for this. If somebody came up to you 10 years ago and said, here is somebody who has 1.7… who makes errors 1.7 more times than any of your engineers.
Bryon Mackay: Would you hire this person? You'd be like, no. But then you'd say, what if they could code 100 times faster? You'd be like, well, maybe, you know? So it's this funny, like, what kind of world we're living in.
Bryon Mackay: But yeah, a lot more issues. Here's some more issues.
Bryon Mackay: Performance regressions are 7x more likely. Readability is 3X, has 3X more issues. Security vulnerabilities, that's a big one. 2.74.
Bryon Mackay: I mean, we're talking… these are some serious numbers here of more issues that could be coming out, and we don't even know, what, you know, what's going on. Like, we don't even know why, initially, because we probably didn't read the, you know, I'm not saying all of you, but I'm imagining a lot of us probably didn't read the code thoroughly.
Bryon Mackay: Error handling gaps, two times more common. Logic and correctness, 75% more common. Okay, this is the kind of stuff that's getting out there.
Bryon Mackay: Okay? This is the kind of issues that we're seeing. AI and LLMs are not doing a better quality job. They're just doing it a whole lot faster. And that's what we're real… that's what we've been focused on for so long, right? 10x engineer!
Bryon Mackay: Right? We hear that all the time.
Bryon Mackay: What does that really mean? It's just, they're just coding faster, but they're actually coding worse.
Bryon Mackay: So…
Bryon Mackay: This is where we come in and go, okay, let's figure out what's going on. What can we do about this?
Bryon Mackay: All right. The first point here is, I think, pretty, pretty evident to all of… to any of us who's tried to do this, especially at a production level.
Bryon Mackay: Right? We need to make sure there's a lot of context. So…
Bryon Mackay: They have no knowledge of your business rules, nowhere of architectural constraints. Sure, Cloud Code will go through your codebase and try to understand all these things. It will do its very best, and quite frankly, it does a really impressive job at that.
Bryon Mackay: But it's still missing constraints. It still doesn't know about this function that's buried over here that could actually be really helpful in this situation, right? There's a lot of things like that that we as engineers for years and years and years prided ourselves on… ourselves on, on knowing, because we wrote the code by hand, and so we knew all the ins and outs.
Bryon Mackay: No understanding configuration patterns. This last one, is something that I want to, highlight here, and that is that a lot of times, and I do this, I think we all probably do this to some extent.
Bryon Mackay: we will keep a Claude session open, or a codec session open for so long that we are just… we just have exceeded the reasonable level of context in the context window.
Bryon Mackay: So, context window currently is, what, a million for Opus, right?
Bryon Mackay: Okay, a million does not mean we should use a million tokens.
Bryon Mackay: Right?
Bryon Mackay: I would stop at $100,000.
Bryon Mackay: If we get to 100,000, stop, consolidate, do something, you know, start a new session, whatever it takes. But by that point, there are studies that are showing that it is starting to lose its ability to remember things at that point.
Bryon Mackay: Okay, so that's something that we have to be, really cognizant of as well, is how much context are we using? And I'm guilty of this. I have context windows that are…
Bryon Mackay: super bloated that I just want to keep going back to, because it went so well last time, but it inevitably, will cause more… well, not to say more issues, but it'll just… yeah, it will be more issues. It'll come at a more… at a more frequent clip, right?
Bryon Mackay: And so, that's something we have to be, really keen on, really aware of, as we… as we work through our code.
Bryon Mackay: But here's the bigger thing.
Bryon Mackay: We do not have the time to review all the code that an AI produces. Now, I want to challenge you here for a second, because every company will say at some point.
Bryon Mackay: We want more code. Look at these things, they go so fast. 10x engineer, yadda yadda yadda yadda, right?
Bryon Mackay: What if we said no?
Bryon Mackay: What if we said we're not doing that? What if we said we're actually going to go 2X faster, or something to that extent, right? Before AI, issues arrived at human speed, we could review them, right? You had your engineer sitting next to you, you'd write code, he'd write code, halfway through the week, you'd swap code and read each other's code.
Bryon Mackay: You know, that was manageable, right?
Bryon Mackay: Now, things are coming out so fast, oh, you can, you can, you know, essentially vibecode your way through a problem, and it's, like, you just keep pushing out features. I had…
Bryon Mackay: I had a situation where I had a junior engineer sitting next to me, and I asked them to go do some task. And he came back with a 300-line PR.
Bryon Mackay: And I looked at the PR, and I mean, about 20 lines in, I go, this could all be done in about 3 lines.
Bryon Mackay: And so I went back to him and was like, hey.
Bryon Mackay: you've… here's your solution. I'm just gonna… I'm just gonna give it to you and tell you what it is, because what you have is way too much code.
Bryon Mackay: It's 3 lines, to do something that he had, essentially had an AI bill 300 for. So there's these checks, right, that we need to be aware of. I'll say this again later.
Bryon Mackay: But…
Bryon Mackay: the bottleneck that we should experience, we all… you know, what is the bottleneck? Well, it's reviews, right? No. The bottleneck is our ability to ensure quality.
Bryon Mackay: And however that happens, happens.
Bryon Mackay: But that should be the bottleneck.
Bryon Mackay: Right? So, we have to be really, leery, and even push back when we're being pushed to do things to our… and that might be a hard thing for some leaders to hear, but as engineers, it needs to start with us.
Bryon Mackay: So… Alright, so what do we do here?
Bryon Mackay: Where do we start? So let's say we want to make sure that we talked about context, we've talked about quality, right? And assuring quality. So where do we start first?
Bryon Mackay: Pseudocoding the logic first is a great way to go. Doesn't have to be pseudocode in your codebase, but it could be something where you are, where you talk it out, where you really think deeply about how is this going to work. I really find it valuable
Bryon Mackay: To tell… to build out documents that… where I, like, go through the logic myself, and I go, this is how I expect it to be built. Like, that's kind of my practice, too, with a lot of PRs, like, I expect it to be built this way, let's see how they end up doing it.
Bryon Mackay: But you provide that kind of guidance to the LLM, and suddenly it knows exactly where it should go, right? Us as engineers need to be aware of our codebases. We are aware of our codebases, and so there's a lot of guidance that we should be providing, as opposed to saying, hey, go make this button, and make it
Bryon Mackay: You know, make this button so they can change their status.
Bryon Mackay: go through the steps a little bit more. How do they… how does this button, you know, how do you want this button to send the API request? How do you want that API request to get to your service layer? How do you want that to go to the database? Should you make…
Bryon Mackay: And just be a lot more, a lot more detailed in your context that you provide. Don't let the LLM decide, because heaven knows, as we've seen in those numbers, it will do a poor job.
Bryon Mackay: Of course, this goes back to research,
Bryon Mackay: And I'll go into this a little bit. I do have a practice that I really like doing when it comes to doing research. And it's not really research as much as it is having a team architecture conversation. So what we'll do is we will get the engineers in a room, the engineers that are working on said task.
Bryon Mackay: we'll turn on Notion. Notion is my…
Bryon Mackay: transcriber of choice, personally. I just… I like it. There's lots of great transcribers, but that's the one I like to use. Turn on Notion, and we'll just start talking for an hour or two.
Bryon Mackay: About whatever the, whatever the, you know, task is at hand. We'll go into a lot of detail, like I was talking about. We'll go into a lot of detail of how to get
Bryon Mackay: this task completed, and the changes that need to go into making it. Not saying to do this for everyone. Of course, there are small tasks that you can get by with without this much rigor, but for larger tasks, it's important for us on the team
Bryon Mackay: to know all the innards of what's going on, right? I don't want to be an engineer on the island. I want to make sure my team's aware, and we have these conversations, so that we can build this context up.
Bryon Mackay: When we're done with our conversation, and it's all transcribed in Notion, we go through the transcription… not the transcription, it usually… we then take that and say to Claude, hey, go take this transcription and build out, a spec list.
Bryon Mackay: And filled out some tasks. And then we go through those meticulously.
Bryon Mackay: And make sure that those align with what we're doing, and then we can go to the AI. Once we have those edited, fleshed out, and we're feeling really good about where we are, then we go to, you know, COLD Code, or Codex, or whatever you're using, and implement it. But it still just takes a lot of thought, and a lot of engineering know-how.
Bryon Mackay: To get there. So, when I say pseudocode logic first, there's also a lot of planning involved. Of course, there's research involved if there's things you don't know. But there's this collaboration between engineers, too, that goes a long, long way.
Bryon Mackay: Okay.
Bryon Mackay: There's a couple other things you can do, and this is more on a team level.
Bryon Mackay: In terms of what you can accomplish. So, context files are super helpful, especially when you have teams, when you have a repo that you have many, many people working on, and they all have different,
Bryon Mackay: I'd say mastery, understanding of different areas of it, of your codebase.
Bryon Mackay: So you have these CloudMD files. Codex has one as well, I think they're called Agents.md, I can't remember the exact name. But you basically can put these to set rules and constraints and patterns and understandings so that you're not relying on the LLM to go through your codebase and understand what's going on every single time.
Bryon Mackay: Instead, it will rely on these files to go in and understand the rules and the constraints of a given file.
Bryon Mackay: A good example of this would be, to, to say, don't, you know, you're not allowed… you're only allowed to change authentication logic in this one spot. You cannot put authentication logic anywhere else in the codebase, it has to go through this file.
Bryon Mackay: Right? That will stop the LLM from going out and trying to do some things that are just not safe, or try to bypass some security measures that you've already established. So those are the kinds of things that you want to put inside those context files to give that LLM, that coding agent.
Bryon Mackay: It's, it's guidance so that it does not go off the rails and start to cause those security issues or other performance issues that we were talking about earlier.
Bryon Mackay: Okay?
Bryon Mackay: Centralizing risk, something I just kind of mentioned in that example, but making sure there's, a single owner, mainly like the files, right? Everything's consolidated, it's in one spot, and anytime those change, that should raise a flag, right? You can set these triggers here and there, to say.
Bryon Mackay: You know, when this file changes, we need to get a review from so-and-so, or we need to get a review from this guy, or that guy, or from platform, or from security.
Bryon Mackay: You know, those things are all really, really good processes to have in place to make sure that the code that we're writing and the changes that we're making, even if it's a junior to the most senior, they're all getting reviewed by the same people. We all know what's going on.
Bryon Mackay: Right?
Bryon Mackay: Smaller PRs. Now, there's always give and take when there's a number involved here, especially something like lines of code, but…
Bryon Mackay: It's more of a code smell, I'll admit, but if you're getting around the 400 line mark, or getting above it, especially significantly.
Bryon Mackay: Think about it. Think about it for a second. Why is it taking so much code? More likely than not these days, the coding agent is bloating, right? They're usually going out there and they're just writing a whole lot of code to get something done, and it's a good time to stop and be like, wait a minute, what's really going on here?
Bryon Mackay: Your PR reviewer will thank you too, as you can probably imagine.
Bryon Mackay: The 7 file change, again, it's not really a hard number, it's a bit arbitrary.
Bryon Mackay: But…
Bryon Mackay: It's a… it's kind of that rule of, like a code smell. Again, if you're changing 15 files, you're going, wait, wait, why are we doing so many file changes? Now, sometimes you have to, right? Sometimes there are cases where you're actually just, you know… but you know what those are. I'm not trying to… I'm not trying to simplify… oversimplify this a bit.
Bryon Mackay: But again, it's just good things to keep in mind. Like, LLMs are great at writing a whole lot of code.
Bryon Mackay: And this is just one of those opportunities where you can sit there and go, when I see a PR with, like, a thousand line changes, I'm gonna stop and go, wait, what's really going on here? Why do we need all of these line changes? Let me understand this better.
Bryon Mackay: Another thing that I've seen a lot in the code from our challengers, which is actually quite surprised me, is naming. There gets to be a lot of generic names that LLMs will throw into code, and that can be problematic when you try to read it.
Bryon Mackay: Generic names, or using, you know, small shorthand.
Bryon Mackay: There's just… there's just no room for that. And the way that we can remedy is quite simple, just tell… either… either you name it yourself.
Bryon Mackay: Or just remind the LLM to be explicit and see how that goes, but I would lean into you naming it yourself when it's really critical, and when it's important, so that you can make sure that when you come back and read it, you know what's going on.
Bryon Mackay: Right?
Bryon Mackay: Finally, checkpoint commits, right? Commit at logical checkpoints, these are all things that are really useful.
Bryon Mackay: And one tool that I… I'm not affiliated with them, but I'm just gonna throw it out there. If anyone has used Graphite.
Bryon Mackay: Graphite is an excellent tool for this kind of thing.
Bryon Mackay: Reason being is because what they… I'll just really quickly touch on it. It can sound a little confusing if you're not used to this kind of thing, but basically they'll make a stack of PRs.
Bryon Mackay: Where every PR relies on the previous PR, right? It's built on top of itself. But they have some really great functionality around
Bryon Mackay: If you have to make a change at, like, the bottom of the stack, that stack, that then gets propagated to the rest of the stack. Really, really helpful, when it comes to, when it comes to, like, building small PRs and building them in
Bryon Mackay: And kind of some sort of scope, where it's like, hey, these are going to be my database changes, then it's gonna be my service layer, then it's gonna be my API layer. Really, really helpful, when it comes to putting all those together, so…
Bryon Mackay: If you haven't looked into it, I recommend it. It's not for everybody, and it's not for every use case, but I think graphite and the process they have is, super helpful, so…
Bryon Mackay: Okay.
Bryon Mackay: This… this is another thing that I've seen so much of, and I'm totally guilty of this, and I'm learning better ways now.
Bryon Mackay: It is really easy for LLMs to make tests, and we all feel so good about it, because we're like, look.
Bryon Mackay: I'm running tests, and this might be the first time in years that I've really written this many tests. But the fact of the matter is, is that these tests are usually low value.
Bryon Mackay: My favorite one is when they're verifying that a static map is indeed that static map, and it's like, no, we don't need to test that, like, this is a string, we don't need to spec that this value is here, because
Bryon Mackay: That's just what it is. So, there's a lot of tests that it'll write that are low value. Now, I'm not saying there's no value in them, okay? There's value in them, and we should. We should be having them write tests.
Bryon Mackay: But there's an opportunity here for us to consider what other tests we might want this, to verify that this is actually working.
Bryon Mackay: And I'll give you a couple examples of categories. We've got edge cases, right? There's a lot of edge cases we can test. Security is another one of those areas where we can make sure that, yes, a test for this security thing is actually there. One thing that you have to be a little more mindful of sometimes with coding agents.
Bryon Mackay: is verifying parameters, right? So, sometimes I've seen coding agents where they'll automatically do it, other times they do not, and they really should.
Bryon Mackay: And then those are things, of course, that you would test. So, there's… I would almost… I would… I like to think that thinking through the tests are just as important, and this is a little… this has been said for many, many, many years, especially if you're in a TDD environment.
Bryon Mackay: That we should also be thinking about the test just as much as we're thinking about the contact… the plan, the planning phase of… that we eventually hand to the AA agent.
Bryon Mackay: It ensures not only that the, right
Bryon Mackay: That the right functionality is there, but it also ensures that there's no regression later.
Bryon Mackay: You may be doing this already, but something that I've almost forgotten to do, because I just have it… I have it drilled into our skill, is after every change, that it goes through and runs all the tests.
Bryon Mackay: That it runs through them and checks for regression. And it's amazing, and kind of scary.
Bryon Mackay: how often regression tests are failing because of something that the AI just wrote.
Bryon Mackay: So, again, really important to have that set up, to have your tests set up. It's really important to make sure that your tests are covering the proper things, and the things that, are critical to your business.
Bryon Mackay: Also important that you give the LLM, or I keep switching off between the two, but you know what I mean, the coding agent, the understanding and ability to go and run those tests every time.
Bryon Mackay: So, I would just put a skill. Personally, what I do is I just… well, I put it in the… I think we have it in a skill, and we also have a CloudMD file, where we just say, after every change, when I say it's all done, like, always run the test before, you know, before committing.
Bryon Mackay: And make sure that that's there. And of course, we have those things already also built in as pre-commit hooks.
Bryon Mackay: But easier to have the LLM run it, and then go back and change it, and work through the problems themselves, and having to go back and forth.
Bryon Mackay: That also is true for UI changes with, things like Playwright.
Bryon Mackay: If you want to include that as well, it just is one less step that you have to do. Of course, now they all have their own… they're all coming out with their own,
Bryon Mackay: Browser use so that you can go in there and… and simply say, hey, go spin up the app, go make sure it actually runs.
Bryon Mackay: I would have loved to have had that, you know, a year ago, when I was doing some heavy UI changes to have it go figure that out, so I wouldn't have to be the one to do it, but… So anyway, a lot of great tools are coming out, and more still to come. Like I said, I still think we're in the early, early stages, and we haven't quite figured out where all this is going.
Bryon Mackay: But when we do, it's gonna be… it's gonna be a heyday.
Bryon Mackay: So,
Bryon Mackay: Okay, I think we know this, we've been drilled with this in our heads as engineers for many, many years, so won't say any more on that.
Bryon Mackay: Next, so we have, guard rails, are really…
Bryon Mackay: it's… guardrails is such a funny term, I feel like, because it's… sometimes we say it to say, like, oh, you have to make sure your app has guardrails for all these things. I don't know, it just…
Bryon Mackay: It's a very vague term in terms of what actually it is, but we're gonna… so we're gonna talk about what kind of guardrails you can put in place,
Bryon Mackay: And we can… and then you can take that back to your work and see what, how you want to implement it.
Bryon Mackay: But the first… is skills.
Bryon Mackay: So, skills is an excellent, excellent way of defining a process that you want, the LLMs to do.
Bryon Mackay: And they're super useful. So, for example, I, as part of my job, I pull down a lot of code, that I have no idea what's inside, right? I am pulling down code all the time and running it on my machine.
Bryon Mackay: And so one thing I do is I have a set of skills that goes through the code and just makes sure that there's nothing nefarious in there, nothing that's gonna hose my machine.
Bryon Mackay: And it's just… it's just part of the regular workflow. I've forgotten it's there, quite frankly. The only time I remember it's there is when I go, wow, this is taking a little bit longer than I thought, and then I look and it's, like, still looking through security concerns.
Bryon Mackay: So… but it's great, because I never have to think about it again, and it just takes one more thing off my plate. So, if you have any of those, and it could be, let's see, I'll give you a couple examples here.
Bryon Mackay: we had one that would check our Click House instance to make sure that things were… and then if there was an issue there, it would go and, update, or try to scale up our nodes, or scale down our nodes.
Bryon Mackay: We had other ones that would do document generators, that was a really, really useful one, so whenever anything changed, we had a skill for that. But there's just all sorts of things where it's like, we always forget, right? So write a skill that just does it for you, and you don't have to remember as much, which is nice.
Bryon Mackay: Okay.
Bryon Mackay: So… Linters.
Bryon Mackay: I still am a big advocate of lynching. It's static, yes. It's not AI, yes. But it's still really great, and it's a great thing for the LLMs, to be aware of. So, again, as part of your either pre-commit, or as part of your skills, or your CloudMD files, it's another great thing to put in there. It's like, hey, every now and then run this.
Bryon Mackay: Make sure that, you know, when you change, when you're done changing, you know, 400 lines of code at a time.
Bryon Mackay: go and run the linter and see where we're at. It'll catch a plethora of issues and mistakes well before, it hits production, and it just gives, again, the LLM a chance to self-correct.
Bryon Mackay: So I'm still a huge, huge fan of lenders. I know there are… there's a lot of opinions out there about what lenders should do, but regardless, I think there's still a lot of value there.
Bryon Mackay: Okay.
Bryon Mackay: So, you can choose, as well, what models you're using for different tasks. Now, it's very easy for me to say, hey, I'm gonna use Opus, because Opus is the best, and it is.
Bryon Mackay: Try to convince me otherwise.
Bryon Mackay: But, sometimes we don't need it, right? However, when it comes to the critical gates that we have in our system.
Bryon Mackay: That is where we want to use the best model possible.
Bryon Mackay: Right? Before it goes out to production, when we have a coding agent go through and review our code, you better believe that we want the best thing possible, because we want to catch everything we can.
Bryon Mackay: So…
Bryon Mackay: Be mindful, right? We all have tokens, there's a cost, it's super easy for an LLM to become the most expensive thing in the engineering, budget.
Bryon Mackay: But, so be mindful of that, but also when it comes to critical things, like, use the smartest model you can get your hands on for that verification.
Bryon Mackay: Or, if you're using a lesser model, just make sure you give it enough context and enough guidance so that it can do a good job as well.
Bryon Mackay: But I wouldn't leave that up to a weak one, and I think that's… I think that's pretty…
Bryon Mackay: Pretty obvious, but worth saying.
Bryon Mackay: Okay.
Bryon Mackay: Finally, separate the loops. Keep the verification process distinct from the coding process. So,
Bryon Mackay: this is… this is where you want to make sure that, you know, you still have other eyes on your code, right? You have other processes that are going through and verifying that, yes, what you have written is going to work in production. That can be QAing, that can be,
Bryon Mackay: That can be, you know, other agents, that can be different… there's a whole lot of ways that we can go around this.
Bryon Mackay: I would also want to say that verification does not happen always after the code's been pushed.
Bryon Mackay: So, to give you an example.
Bryon Mackay: We had, we had a case where we had to move data from a Postgres database to an OLAP database, because it was all analytical data that we wanted to aggregate. There's a lot of numbers that we wanted to crunch.
Bryon Mackay: And, the verification process actually started in the planning phase.
Bryon Mackay: So my co-worker, dutifully, did his research, went through the whole thing, and wrote this really great planning doc.
Bryon Mackay: And it was my opportunity to then verify it, the plan. And so I went through it myself, I read it, I made some suggestions, most of it looked… it pretty much all looked great, there were just some additions I would have included, and then he went and built it, and it was… it was great. It was very, very smooth.
Bryon Mackay: But again, we didn't rush it, right? He wrote it, and then I verified it, and then he wrote the code, and then I verified the code, and we had coding agents reviewing the code.
Bryon Mackay: And it was a very, very smooth process. But you want to make sure that, of course, that you have
Bryon Mackay: you know, multiple people look at that. If you've ever studied the, or looked into,
Bryon Mackay: Oh, now I'm gonna forget the name. I wanna say Systems of Control, but I don't think that's quite the right term.
Bryon Mackay: Where, basically, you can't… you don't give one person all the keys to handle finances, right? You have multiple people that have different keys, that have different gates that they're in charge of. Still want that, still need that. Does it slow things down? Sure. Sure, it does.
Bryon Mackay: But we shouldn't let that speed be the determining factor on what we do.
Bryon Mackay: Which leads me right to this thought.
Bryon Mackay: If any of you know DEX, from Human Lair.
Bryon Mackay: Segregation of duties. Jason. Yes, I think that's… I won't… I don't know if that's the exact right term, but that's really close to it.
Bryon Mackay: It might even be the right one. It's been a while for me since I've had to use that term. Anyway…
Bryon Mackay: This is from Dex, he's at Human Layer, you may know him, he's pretty… he's getting pretty big on… on X and other places. But he said this the other day, and I just… I think this is…
Bryon Mackay: An excellent frame of mind to have as engineers.
Bryon Mackay: He said the bottleneck is your ability to ensure quality.
Bryon Mackay: I think that's so true.
Bryon Mackay: I think that is absolutely what should be the bottleneck. I know we all want to push things fast, but for heaven's sakes, slow down.
Bryon Mackay: And we should be pushing that up the leadership as well. That, look.
Bryon Mackay: We're engineers, we're gonna make sure things work, we're gonna get more pushed out than ever before.
Bryon Mackay: It may not just be the 100X that some people say.
Bryon Mackay: It's not gonna be us open-clawing our way through, it's just not gonna work that way. And… if Anthropic's, little PR
Bryon Mackay: Snafu that they had this last week.
Bryon Mackay: Where we all got a peek a little bit into what Cloud Code actually looks like.
Bryon Mackay: As well as that product works, there's some raised eyebrows. So, let's not be like that.
Bryon Mackay: And let's make sure that our quality is good. It's really… it's almost becoming a bit of a moat, in a sense, these days. I don't know how long that will be.
Bryon Mackay: But to have a quality application that's secure.
Bryon Mackay: It's always been a mode, I suppose, but even more so today, with all the things that are going on, those metrics that we saw earlier kind of allude to that.
Bryon Mackay: That we need to make sure that the quality of our code is still the most important thing that we can do with our jobs.
Bryon Mackay: I do want to point out one other thing here, with this.
Bryon Mackay: I saw an article the other day.
Bryon Mackay: Excuse me. That I thought was actually… the headline was kind of funny, but also, said something about our jobs as engineers.
Bryon Mackay: It said, CEO fires all lawyers and relies on ChatGPT for his lawsuit. Loses $250 million case.
Bryon Mackay: And I just thought.
Bryon Mackay: We as engineers are like those lawyers, and sometimes people feel like they are going to vibe their way out of a problem.
Bryon Mackay: Because, look at me, I can build a website. And it's like, cool. Do you know what's going on in that website? Do you know what's going on under the hood? And there's a lot of opportunities for us as engineers to step in and be like, look.
Bryon Mackay: We know what we need to do, we know what we need to put into this. We are going to ensure that quality does not lack.
Bryon Mackay: We're going to ensure that your app is secure. But to do that, we need to slow down. We'll still build more, but probably at a 2X pace. And quite frankly, most companies need to be okay with that, in my opinion.
Bryon Mackay: Yes, we can build feature after feature after feature, but as consumers, can we even consume that many features?
Bryon Mackay: Not really.
Bryon Mackay: Not really. We're creatures of habit, and if we're breaking the habit too much, what's going on?
Bryon Mackay: But there's something to be said about quality. I feel like right now we're in a bit of a state where quality is a moat. If you can ensure quality.
Bryon Mackay: You have a moat. You're gonna be just fine. You're protected, at least for the foreseeable future.
Bryon Mackay: Call me back up when we have Opus 5.7, or whatever it is that comes out, and we'll see what happens then. But I really don't think it's gonna change too much.
Bryon Mackay: Alright, this is a little bit for non-engineers. I don't know how many people in here are non-engineers.
Bryon Mackay: But I just want to point this out.
Bryon Mackay: that, as a non-engineer who's trying to build, their exposure is higher. They do not understand the ins and outs, as engineers do.
Bryon Mackay: But engineers… so that when they build something, they're not aware of all the things. As one person likes to… as one phrase I like is the unconsciously incompetent.
Bryon Mackay: Right?
Bryon Mackay: Where they just don't know what they don't know.
Bryon Mackay: So, it's very important.
Bryon Mackay: to have an… at the very least, what I would recommend is sit down with an engineer and have them help you set up your workspace. They can put together a certain structure using… you can leverage CloudMD files, you could set rules and guardrails in there.
Bryon Mackay: You can do… you can set up skills, and those kinds of things will help a non-engineer go a lot further and a lot safer. So, for example, if you have a PM that wants to do some changes, or you have a designer that wants to do some changes.
Bryon Mackay: say, hey, let me set up your workspace, but let me only give you access to these files. Like, here are areas where… here are… we're gonna blacklist a whole bunch of this application, or whitelist some of it.
Bryon Mackay: And so that they have the ability, right? They could very well do a lot of meaningful work.
Bryon Mackay: But, we don't want to give them the keys to the kingdom either.
Bryon Mackay: Which is what seems to be happening at some places.
Bryon Mackay: So, anyway
Bryon Mackay: Finally, and to that point, the next point here, never deploy anything touching auth payments or user data without a review, right? Think about blast radius, and just what awful things could happen, when people do… when people put those things together and, unknowingly, right? And then they release it, because it gets through some review process.
Bryon Mackay: And suddenly we have… Bigger issues on our plate.
Bryon Mackay: I want to just come back to this slide and leave this up here as I say one more… as I kind of talk a little bit about what Gauntlet's trying to do here. So, here at Gauntlet, like, this is… this is our… this is my job, this is our job, to help other engineers
Bryon Mackay: know how to set up codebases, know how to be the AI leader that a lot of these companies need, right? We need people with good frames of mind where they think about these problems in terms of.
Bryon Mackay: what's the worst that can happen, and how do I prevent that? There's also people… we also need to have good system designers, right? We have to understand the systems that we're in, that we're working in.
Bryon Mackay: The, we have to understand the libraries we're leveraging.
Bryon Mackay: And how those make for a better, you know, how we can leverage those to make a better application.
Bryon Mackay: and avoid overloading our, you know, all of the dependencies that we may have. We understand as engineers these, these qual… the qualities of a good app.
Bryon Mackay: it's getting harder and harder to do that when it's so easy to vibe. So what do you do?
Bryon Mackay: Did we just stop vibing? Did we stop using AI? No, of course not. We're gonna keep using this. But we have to use it strategically, we have to use it well, and we're fearing, I'll just give you a little insight.
Bryon Mackay: When we talk to the companies that hire our graduates, the things that I'm hearing from them, resonate there, where there are still some that want more features, and that's just kind of what… how they come off. I don't know if that's entirely what's in their mind.
Bryon Mackay: But they come off as saying, like, yeah, I want these engineers to work faster and build more and more and more, and that's great, right? That's okay.
Bryon Mackay: But, it also comes with a caveat. We're teaching them as much as we're teaching those that come to Gauntlet.
Bryon Mackay: That there's a lot more nuance there that you are not… that they're not aware of, and we have… we'll supply… we'll teach our engineers how to identify that nuance, how to be a good communicator of that nuance, so that it can be addressed appropriately.
Bryon Mackay: So, a lot of what we teach, and with every cohort, this changes, right? Every cohort gets a kind of a new flavor of it, because in 10 weeks, so many things can change, and so many things are becoming apparent.
Bryon Mackay: That, so…
Bryon Mackay: where was I going with that? The next time we do this cohort, the curriculum's gonna change a little bit. And we're gonna make sure we focus on some issues that we're seeing pop up all over the place. That's a big thing and part of my job, is to make sure that we're staying up to date.
Bryon Mackay: And then, when it comes to… so, anyway, all that's… if you are interested in coming to Gauntlet and learning more about how to, be that kind of leader in a, in an…
Bryon Mackay: in an environment where AI is working, it's thriving, but it's also very dangerous, and you want to be the voice of reason to help make sure that these companies actually
Bryon Mackay: Produce quality output.
Bryon Mackay: Come talk to us. It's a great opportunity. It's 10 weeks, fully paid for on our dime, not yours, and and you'll have the opportunity to interface with these companies and get jobs with them.
Bryon Mackay: We have cohorts, starting throughout the year, and so we have another cohort starting in about three and a half weeks. So, if that's too fast for you, though, we have other cohorts starting later.
Bryon Mackay: In the year. And we'd love to have you. We'd love to have you. It's such a great time. You get, there's kind of 3 things that I always point out to people that you get a lot of Gauntlet. You get, of course, the learning, and the opportunity to just work on your craft.
Bryon Mackay: We always talk about 100-hour work weeks, that's not an exaggeration. There's a lot of work
Bryon Mackay: Excuse me, that we're due every week.
Bryon Mackay: And it's just a great time to hone your craft and really dig into this stuff without any other distractions. Two, you get to interface with a lot of great companies, so you get to see… you get to meet great people, you get to talk to them about what they're building, and decide if that's a good fit for you, and if that's something that you want to go do, in the future. And of course, all these companies are offering very high salaries, and so it's a great chance to kind of bring them all together, all these companies together, all these great
Bryon Mackay: Engineers together, and, to make some, some great, some great jobs happen.
Bryon Mackay: And then the third thing, which is a little bit in line with number two, is networking. You not only get to network, though, with the companies.
Bryon Mackay: But you get to network with a great group of people that come in with you into the cohort. You're all like-minded, you're all engineers that want to learn more, you want to push the envelope, but you want to do it in a way that is… that produces value and quality.
Bryon Mackay: And that's what you get when you come to Gauntlet. A great time to sharpen that saw. We'd love to have a lot of people out in Austin.
Bryon Mackay: Where you get to be there for 7 weeks, and you just get to really work and grind. It's a fun, fun time, and a great way to get to know people, and also, and also just land that great job.
Bryon Mackay: So… Alright.
Bryon Mackay: I've been talking a lot, I'm gonna stop there. Haven't seen any hands pop up just yet, but if you have any questions, whether about the topic we've talked about, or, if you want to talk about, if you want to talk about Gauntlet, I'm here to… to answer. So, I'm going to…
Bryon Mackay: Check real quick here… Let's see… Let's see here…
Bryon Mackay: I had… let's see, guys, I had a document pulled up that I think I might need…
Bryon Mackay: Oops, not that one. Sorry.
Tom Babb: Also, we have someone with… that raised their hand, Christian, I'm gonna allow it to talk.
Bryon Mackay: Oh, okay. Oh, I didn't see it. I'm sorry there, Christian. Go ahead.
Tom Babb: I've just done it.
Christian Jarjouhi: Hello.
Christian Jarjouhi: Can you hear me?
Bryon Mackay: Yep, I can hear you. Go ahead.
Christian Jarjouhi: Yeah, thank you so much for the presentation. It resonates well. I've been doing vibe coding for the past few months.
Christian Jarjouhi: And I realized the amplification of the errors. And, your point resonates well. I'm trying… for… my question is related to gauntlets. I tried to apply, and it looks that…
Christian Jarjouhi: There's, somewhere,
Christian Jarjouhi: that the review will… there will be a review, and there will be… somehow, a committee will be reaching out afterwards. I don't know if that, like, if there's any timeline on that, there wasn't.
Christian Jarjouhi: And second question is, in terms of the teacher or instructors, like, what are the typical profiles, for the, for the instructors in, overall, and is there…
Christian Jarjouhi: some kind of teacher assistance over there, or, just wanted to shed some light on that, on the content process. Thank you.
Bryon Mackay: Yeah.
Bryon Mackay: Yeah, so, first question about, having people, there's somebody… waiting for somebody from Gotland to reach out. I will… I can… we can reach out to our guy. There's a guy here named Drew. He's usually… he's the one in charge of that, and, he should be reaching out to… to see if…
Bryon Mackay: Let's see, have you taken the CCAT already, Christian?
Christian Jarjouhi: I'm sorry, yeah, yeah, I did that. Okay.
Christian Jarjouhi: I believe I was in the 80s something? I didn't reach the 90s.
Bryon Mackay: Oh, like the 90th percentile?
Christian Jarjouhi: Yeah.
Bryon Mackay: Okay, cool. Yeah, so,
Bryon Mackay: I know there's a lot of people applying, so let me… I can… I can take your name and send it over to Drew, but yeah, you should be, you'll hear something from him, whether… either way, to… to see, what's going on there. So, if you haven't heard from him yet,
Bryon Mackay: Hang tight.
Bryon Mackay: But on the other side of it, with the instructors, yeah, all the instructors we have are all engineers. They're either actively working at a company, or they're working specifically at Gauntlet. They all have many, many years of experience. I'm trying to think of… I don't even know who has the least, per se, but…
Bryon Mackay: 10, 15, 20 years is not… is typically what the people have that come and instruct here. So, so yeah, lots of… lots of really good engineers, a lot of the… I respect all of them.
Bryon Mackay: And they are all building great stuff on, when they're not teaching. So, they're very much well-versed in what's going on, and how to leverage the tools to get their jobs done.
Christian Jarjouhi: Thank you so much.
Bryon Mackay: You bet, thank you.
Bryon Mackay: I'm gonna read a few from, the comment, or the, yeah, the comment section here.
Bryon Mackay: Let's see…
Bryon Mackay: Looks like Austin is looking for anybody who wants to meet up at the next AI Camp event at Capital Factory, very cool, that'll be fun.
Bryon Mackay: Igor asked, when do you guys stop taking applications for this cohort? Do you guys do stuff like fine-tuning or RAG? Yes and yes. I will say that RAG is one of those that is ever-revolving.
Bryon Mackay: And it's used in many, many companies. Fine-tuning is, in my opinion, a really great one to know and understand.
Bryon Mackay: But if there's only specific companies that will actually use that skill. More often than not, there's a lot more using of foundation models, and,
Bryon Mackay: And just… and using, you know, kind of the top, top-tier things that are… the top-tier LLMs that are out there to… to get work done, so…
Bryon Mackay: And as far as take… when we stop taking applications, I'm actually not sure, but I know that we accept people almost up to the day it starts.
Bryon Mackay: So, I would… if you wanted to do this cohort, we have, you know, three and a half weeks, I would take… I would say take the time.
Bryon Mackay: you know, think about it, think it through, you have time to get through it. Don't wait too long, though, because there are some things you gotta do. You gotta take a CCAT, and you have to go through a review process, as we've been talking about. So, just be aware that, there's a little bit of…
Bryon Mackay: Just…
Bryon Mackay: Housekeeping that we have to do before we accept you, but we do accept people right up to the date.
Bryon Mackay: Let's see, I'm just looking…
Bryon Mackay: Okay…
Bryon Mackay: I have something else here… oh, you know what? There's a Q&A section that some of you have been putting stuff in, especially you, Jay Fuller, alright. Am I running daily security audits?
Bryon Mackay: I don't know if they're daily, or for my stuff, personally.
Bryon Mackay: moron, but there are certain things that I… there are certain actions I take where I do automatically have security audits happening.
Bryon Mackay: Oh, then you raised your hand. So, there, there's your answer. Apologies. Anyway.
Bryon Mackay: Okay.
Bryon Mackay: Do you also encourage entrepreneurship tracks? Christian, that's a great question. And the answer is, yes, we do. We definitely push more, though, for getting the jobs, but there are definitely times where, if you have this great idea and you've vetted it out, come talk to us. You know, I…
Bryon Mackay: Don't know if you know our CEO, Austin Allred, but he loves that kind of stuff.
Bryon Mackay: And if there's an opportunity there, you… this is a great chance to rub shoulders with him. So, yeah, that's an opportunity as well.
Bryon Mackay: Okay, Jeremy's asking, what kind of person is landing these 500K jobs, staff-level jobs? Creators of startups with 100K app downloads?
Bryon Mackay: Not the creators, necessarily.
Bryon Mackay: there are some people that we've had that had many years at Google, who are very well established already in their communities, in terms of maybe it's an ML or a data science kind of track, or,
Bryon Mackay: What also happens a lot of times is we get some really incredible people that come in, and there are a lot of companies that want them, and then there becomes a bidding war for that person.
Bryon Mackay: So, it… it's not everybody, right? But there's surprisingly more than I thought when I… when I started here, of how many people get up that high. It's… it's pretty… it's… it's impressive.
Bryon Mackay: A lot of these companies we have are willing to pay top dollar for great, great people, so…
Bryon Mackay: Yeah.
Bryon Mackay: Let's see…
Bryon Mackay: Idrisu asked, do you have to meet any eligibilities to apply, like level of experience? Can you apply while still in school?
Bryon Mackay: We have started to consider people who are still in school. If you're taking a CS degree, go ahead and apply, and there's, there's some more consideration that goes into that one, just to make sure that, you're at a
Bryon Mackay: At a spot where you can…
Bryon Mackay: do the program and be successful in it. And, we have some of those happening already, which is so fun to watch.
Bryon Mackay: But it is something that we kind of do on a case-by-case basis. Typically, we require 3 years of experience. That's our baseline requirement. But if you don't, still apply.
Bryon Mackay: And if you're somebody that seems to have the experience, maybe you've coded a lot your whole life, or you've already had a lot of work experience, or whatever the case is, there's opportunities… well, I shouldn't say work experience, that's what we're talking about.
Bryon Mackay: But, if you've had a lot of good experiences and you seem like the kind of person that, could handle the program and also land a job, we want to talk to you, too.
Bryon Mackay: Jordan asks, what kind of salaries are you expecting for Gauntlet, for Gauntlet for America?
Bryon Mackay: Yeah, that's a good question. I would… okay, so there's two here, right? There's salaries and there's whole comp.
Bryon Mackay: So, I haven't done the calculations on the total comp, but the salaries are more typically in the $150 to $170 range, based on when… where… what level you get it at. There's a whole… I don't know if you've looked into this, but there's a whole structure of, like, different… of varying levels when you work for the U.S. government.
Bryon Mackay: But the total comp is pretty high.
Bryon Mackay: Because they do a couple of… for example, there's a pension that you're getting out of that. And and then there's all these other great benefits that they have included in there as well. So, there's a lot…
Bryon Mackay: There's a lot to be said for Gauntlet for America that can go a long, long way.
Bryon Mackay: So, the salaries may not reflect all the value themselves.
Bryon Mackay: But, the perks that you get with that job?
Bryon Mackay: I feel like make it a very viable and reasonable thing to… to look into. Definitely something if you're interested in that. Or if you live in the DC area already, that's where a lot of the jobs will be. That's an opportunity for you. Or if you have security clearances already, that's another thing I've been learning lately, is how important those can be. So…
Bryon Mackay: Great opportunity there, great way to get your foot in the door, and just work on some pretty interesting stuff.
Bryon Mackay: Ryan asked, is the application process different for Gotland for America? No. It's the same.
Bryon Mackay: It's the same. You'll be, if you get a certain score on the CCAT, that's one of our main differentiators of whether you get into Gallant for America, whether you get into, the, the main, the flagship program, or if you don't make it at all. So,
Bryon Mackay: So, yeah. Alper asks, does Gauntlet for America require a citizenship, or would a GC be sufficient? Citizenship is required for that one.
Bryon Mackay: Yeah.
Bryon Mackay: Yep.
Bryon Mackay: Sure, you bet.
Bryon Mackay: Great questions, everyone.
Bryon Mackay: Keep them coming.
Bryon Mackay: Let's see, Jordan asks, do people bring their own monitors to Austin? No need. Unless you really have a monitor you love. No, I'm kidding, don't even bring that. Not worth bringing it. Bring your computer, preferably a laptop, as opposed to a desktop if you can.
Bryon Mackay: But we'll have monitors in Austin that, you get at your desk. So, you'll have plenty to work with there.
Bryon Mackay: And Austin's just a really fun, unique time. Like I said, you get just… it's just time where it's just you and the rest of the cohort, and you're all just heads down, working and learning and pushing. It's… it's a… I mean, what other opportunity in your career do you get something like this, where you just…
Bryon Mackay: you don't have to pay for anything. It's all covered. Your food, your housing, your laundry.
Bryon Mackay: So that you can come and just sharpen the saw, like, for… for however… for 10 weeks. Well, 7 weeks in Austin, right? 10… 3 weeks remote. Then you just do that, and you get a network with great people, you get a high… you get to meet great companies. What an incredible opportunity that is.
Bryon Mackay: So, it's quite the experience. I hope you all strongly consider it, and hopefully we'll see many, not all of you.
Bryon Mackay: One of our cohorts soon.
Bryon Mackay: Alright… Any other questions?
Bryon Mackay: How large is the typical cohort? We typically aim for 100.
Bryon Mackay: 100, 150? Somewhere in there?
Bryon Mackay: We always try to push a little bit higher than that, but…
Bryon Mackay: you know, if we can… if we can get to 200, the material I build is… and the processes I build out on the curriculum side, at the very least, are meant for 200. So we can push that number up if we have that number. But 100, 150 is where we like to start a lot of times.
Bryon Mackay: How soon after graduation do you anticipate starting with a government job? asks Jordan.
Bryon Mackay: That is a really good question, and one that I don't have a solid answer for, because, because this is actually the… we're running it right now for the first time.
Bryon Mackay: And so, there are still some expectations that I haven't gotten clear guidance on. But…
Bryon Mackay: Give me 3 or 4 weeks, and I will have a much better idea of what that looks like. But we are… let's see, we're currently in week 7, and we have got that ball rolling of getting everybody in the GFA program, or Gauntlet for America program.
Bryon Mackay: into their system, so that we can get them, interviewing and getting the jobs they need. So, everything's in motion right now, and they're, being very…
Bryon Mackay: very, easy to work with in terms of getting things moving and rolling. So, hopefully I'll know more concretely afterwards. Ideally, you're starting right after you graduate. Or maybe a week after, so you can get life in order if you need to move, or whatever the case is. But I'm sure they're flexible on that, too. They've got to understand that, right? So…
Bryon Mackay: Idrisu asks, what's the acceptance rate?
Bryon Mackay: That's a good question. I'm… I'm not sure what the exact acceptance rate is. It's pretty low, though.
Bryon Mackay: The requirements we have are pretty high, and there's a lot of review we do with each person, so, it's not terribly high. I can't… I couldn't tell you the exact number, but I know it's… it's definitely sub…
Bryon Mackay: 10%, I would say. Correct me if I'm way off there, Tom, but I believe that's around the number.
Bryon Mackay: Jordan, you're already in cohort GFA5. Awesome. Oh.
Bryon Mackay: You're gonna love it. You're gonna love it. I've been so… I've had so much fun developing this, that track.
Bryon Mackay: I think it's really, really cool.
Bryon Mackay: So, you'll be learning so much good stuff.
Bryon Mackay: Christian, what is the ratio, then, for teachers to students? Good question. Let me… there's probably…
Bryon Mackay: Oh… 10, I wanna say?
Bryon Mackay: There's a lot of us that are just gonna be walking around, talking to you, so…
Bryon Mackay: Yeah, and if there's… that's where we like to keep, and we like to send out teachers to talk to you, and see how you're doing, what questions you have. Lectures, of course, will be there, so…
Bryon Mackay: Quite a… quite… I would say that's a pretty, pretty good ratio. And then don't forget, you have everybody else, too. You're all learning together, everybody comes in with different skills. A lot of times, we have,
Bryon Mackay: within cohorts, there's, like, little meetups that they'll have, where there's, like, for different things. There's, like, a running group, a game group, there's groups that will do lightning talks, and we can help facilitate some of this, but we love it when the cohorts just get together and they're like, hey, let's do this together. It's like, oh, that's great. Because then you're really learning, more about
Bryon Mackay: you know, specific topics that maybe we as teachers, like, don't have time to review or to go into. There's a lot of stuff out there, right? And so it's a great opportunity for all of you to work together and…
Bryon Mackay: Build, you know, build up those relationships, and also just learn some really cool stuff.
Bryon Mackay: Christian asks, and are projects mainly individual in nature, or do we have team projects? You do have a bit of mix, but a lot of it is individual work.
Bryon Mackay: There are cases where you get together with groups to discuss the projects you're building, but a lot of the work has to be done individually so we can assess, your skills. At the very end, I will say there is a capstone where you will get with your team.
Bryon Mackay: And, you'll get with a team and you'll build a product on your own. So, but yes, a lot of it just by nature of having to verify that everybody is capable and able to take on these jobs.
Bryon Mackay: We do a lot of individual work as well, so…
Bryon Mackay: Jason asks, what if I have a day job? It's remote, though.
Bryon Mackay: So, Jason, this is… I'm glad you asked this question. This is, this is one of those situations where you've got to decide for yourself how much to put in. When you come to Austin, you're not going to be able to have that job. Some people are able to take some leaves, others just have to leave the job entirely.
Bryon Mackay: That's typically what has to happen, is people have to leave the job because it's so rigorous and there's so much work to do. So,
Bryon Mackay: Yeah, that's… it's… it's… that's not an easy decision to make, though. I fully understand and appreciate that, that decision, and… but it's… it's kind of one of those things where it's like.
Bryon Mackay: it's… the opportunity is there, it's incredible. So I don't know where you are in your career, Jason, but it's something you gotta consider for yourself, but I… again, I would just emphasize what an incredible opportunity it is to… to go and learn, and just really spend a lot of time
Bryon Mackay: To make yourself the best engineer possible.
Bryon Mackay: Austin is saying, I'm putting in my vacation request as soon as I pass the CCAT. There you go. There you go.
Bryon Mackay: Just… just saving up that time. Hopefully you have 10 weeks. I think the best company I ever worked at only allowed us to have 6, so… You can almost make that work, right? Squeeze it out? Sure, sure. Anyway.
Bryon Mackay: Well, great. Well, everybody, we're, we're at time, but I want to say one more time, just thank you all for coming. Really excited to have all of you here tonight, and just get a chance to talk about
Bryon Mackay: where we are in this crazy AI revolution that we're experiencing. But it's a lot of fun. It's a rollercoaster sometimes. New models come out, and suddenly it's like relearning how to ride a bike, but we love it, and there's so much opportunity out there for engineers like you.
Bryon Mackay: to go and make a real difference and a real impact, especially as all this is happening. We are gonna be…
Bryon Mackay: the… that wave of engineers that really understand and learn this stuff, and solve some really interesting new problems that come up. So…
Bryon Mackay: Thank you all again for coming. I hope you have a… enjoy the rest of your evening, and we hope to hear from you more, and see you at Gauntlet.