Instagram story viewer> @daryleserrant> Posts

Daryle Serrant | AI Systems

@daryleserrant

724

followers

636

following

Understand why AI systems fail
14 years in tech | Pluralsight author + AI researcher
Get my free AI Failure Modes Matrix👇🏾

POSTS STORIES REELS TAGGED

Download All

I've worked in tech for 14 years, but whenever I came across an opportunity asking for experience I didn't think I had enough of, I'd often tell myself, "I'm not good enough."

Sure, I could build a personal project to develop and demonstrate that experience. But I'd dismiss that too. To me, it didn't count unless I did it for a real company.

I wanted to prove I was competent, but I was also afraid that if I pursued an opportunity before I was ready and messed it up, I'd ruin my chances of getting another one.

Eventually, I realized what I was actually doing.

I was hiding.

There was another problem with my thinking: even if I gained the experience I thought I needed at a company, NDAs and other restrictions might limit what I could actually share.

So now, instead of waiting for someone to give me an opportunity, I've decided to create my own. I'm building projects that are relevant to me.

One of those is a personal AI research agent. It's something I genuinely need. It's also an opportunity to deepen what I already know, explore what I don't, and publicly demonstrate what I can build.

I'm discovering freedom in that. I'm no longer putting my development in someone else's hands. I can decide what I want to learn and what I want to build. I can experiment, make mistakes, paint outside the lines, and share what I discover along the way.

I still get those feelings whenever I see opportunities that seem beyond my experience. But this time instead of waiting for someone else to give me permission to become the professional I want to be, I look for ways to create the opportunity myself.

If you're figuring out how to create your own opportunities too, follow along.

I'm figuring it out right alongside you.

#career #tech #ai #softwaredevelopment #justdoit by @daryleserrant

88

21

22 days ago

Before starting this challenge, I already knew how to create content. I had been making videos analyzing AI failures, teaching technical concepts, and sharing what I knew about technology.

But AI content had become an increasingly crowded space. There are thousands of people talking about the same technologies, news, tools, and ideas. I needed a way to stand out from the crowd, and personal storytelling seemed to be one way to do that.

The problem was that I didn't know how personal storytelling would fit with what I was trying to build professionally.

So for 21 days, I decided to do an experiment. I shared those stories anyway.

My first surprise happened on Day 1. I shared one of the stories I was the most nervous about publishing. I expected some people would relate to it. It ended up getting more than 100,000 views.

Telling these stories required me to revisit experiences I hadn't thought about in years. I started noticing patterns in those stories. Many of my stories came back to uncertainty. Separating what happened from what I thought it meant. Understanding the system surrounding an outcome. Breaking difficult problems down. Taking action without knowing exactly what would happen. Learning from the result and adjusting. Those ideas aren't entirely separate from my professional work. They're also part of how I approach diagnosing technical systems.

I'm still figuring out what that connection means for my content and my business, but I don't intend to wait until I have it figured out.

Day 22 will not be the end of the storytelling. I want to keep mining my experiences for stories, adapt more of them for LinkedIn and YouTube, experiment with storytelling in my AI Incident Forensics work, and share more of what I'm discovering as I explore how and why AI systems fail.

For 21 days, I thought I was learning how to tell better stories. I didn't expect storytelling to help me understand the story I'm still living.

#storytellingcontentchallenge @jun_yuh @creatorcollege by @daryleserrant

41

20

a month ago

When I was younger, I thought life worked more predictably than I've discovered it actually does. If I worked hard and became exceptionally good at what I did, I thought the outcomes would reflect that.

Thirty-six years of living have complicated that picture.

I've done good work and still received outcomes I didn't like.

I've been rejected from opportunities and later discovered evidence that made me question what I'd originally assumed that rejection meant.

I've pursued things believing I'd feel different once I got them, only to get them and realize I was still the same person.

I've made decisions I would make differently today, but only because the consequences of those decisions taught me things I didn't know when I made them.

I've also discovered that some experiences I might once have considered detours were teaching me things I needed.

Getting outside the technology bubble, for example, exposed me to people, businesses, and perspectives that ultimately changed how I think about technology itself.

I still believe in working hard. I still care deeply about becoming excellent at what I do. And I'm still ambitious about what I want to create.

I just don't expect any of those things to give me complete control over what happens next.

Maybe that's one of the biggest differences between me now and the person I was when I was younger.

I used to think I needed to figure things out so I could move forward. Now I'm realizing that moving forward is often how I figure things out.

#storytellingcontentchallenge #lifelessons by @daryleserrant

25

9

a month ago

A few days ago, while looking through old emails, I found an email that changed how I remembered one of the hardest periods of my career. It was an email that I had completely forgotten about.

In 2020, I had been unemployed for more than a year after being laid off from my tech job.

I was getting interviews. I just wasn't turning them into offers.

After enough rejection, I started questioning myself.

Maybe I wasn't as good of an engineer or data scientist as I thought.

One opportunity took me through several rounds of interviews for a contract data engineering role. I made it to the end, but they ultimately selected another candidate.

At the time, it felt like another piece of evidence that I wasn't good enough. But several weeks after that rejection, the recruiter contacted me again.

There was another position available. They wanted to know if I was interested, and the email indicated that I wouldn't need to go through another interview.

For whatever reason, I never responded.

Reading that years later made me pause. I had spent so much of that period interpreting rejection as information about my ability. But the same interview process I'd remembered as evidence that I wasn't good enough had apparently left them comfortable enough with me to consider me for another position without putting me through another round of interviews.

“I didn't get the job” was the outcome.

“I'm not good enough” was the interpretation I gave to the outcome.

I wish I'd understood the difference.

I also wish I'd realized how much agency I still had during that period. I could have kept applying while building projects, developing skills, sharing what I knew, meeting people, helping businesses, teaching, researching, or exploring other ways into the career I wanted.

If I could give my younger self one piece of advice now, it would be:

Keep looking for the job. But don't wait for a job to keep building your career.

#storytellingcontentchallenge #unemployment #joblayoffs #careers #tech by @daryleserrant

29

10

a month ago

For a long time, I had an image in my head of what an expert was supposed to look like.

They knew what they were doing. They had enough experience to handle whatever came their way. And they didn't make silly mistakes.

So when opportunities came along that required knowledge or experience I didn't think I had, I sometimes hesitated. I believed I needed to become more prepared before I could start.

One experience that challenged that belief happened when I was developing one of my earlier technology courses. There were topics I was expected to teach that I didn't feel knowledgeable enough about. Part of me was afraid I'd make a mistake or teach something incorrectly.

But I took on the project anyway. When I encountered something I didn't understand deeply enough, I researched it. I studied until I could explain it clearly enough to teach someone else.

I later realized that being relatively new to some of the material had an unexpected benefit. I could still remember what it felt like not to understand it, which helped me explain some of those ideas from a beginner's perspective.

I've experienced the opposite while teaching programming to children. After you've done something for years, certain concepts become so intuitive that you can forget why they're difficult for someone encountering them for the first time.

None of this means expertise doesn't matter or that we should take on work we're completely unqualified to do. It just changed what "being ready" meant to me.

I'm realizing that I was trying to get the benefits of experience and expertise without paying its cost. Real experience includes getting things wrong, encountering problems you don't know how to solve, and discovering gaps in what you know. Sometimes you might even get burned in the process.

Those experiences can become part of what eventually gives your judgment depth.

So I'm now learning to prepare without expecting preparation to eliminate uncertainty. Perhaps real readiness is knowing that when something doesn't go right, I can learn, adjust, and keep going.

#storytellingcontentchallenge #overthinking #analysisparalysis by @daryleserrant

46

26

a month ago

I've spent a lot of my career trying to better understand how complex technical systems actually work.

Not just how to write the code or train a machine learning model, but how the pieces fit together.

These are five books I've read over the years that helped shape how I think about complex technical systems:

Machine Learning Engineering in Action — Ben Wilson
Helped me think more carefully about what we're actually trying to build, why we're building it, and the realities of ML projects from beginning to end.

Designing Machine Learning Applications — Chip Huyen
Helped broaden my understanding of the fundamentals behind building production-ready ML systems.

Designing Data-Intensive Applications — Martin Kleppmann
Helped me zoom out into the software and data infrastructure that complex systems depend on.

The Signal and the Noise — Nate Silver
Made me appreciate just how much uncertainty exists when we're trying to use data and models to predict what happens next.

Debugging — David J. Agans
Gave me practical principles for systematically investigating technical failures. One of those principles helped me solve one of the hardest software/hardware problems I've encountered in my career.

And then there's Thinking in Systems by Donella H. Meadows

I bought it back in 2013 because I wanted to understand how the systems around us work and how we can change them.

I never finished it.

It's funny looking at that book now because more than a decade later, understanding systems—and especially understanding why they don't behave the way we expect—has become increasingly central to the work I'm pursuing.

So that one's going back on my reading list.

#storytellingcontentchallenge #ai #datascience #machinelearning #softwareengineering by @daryleserrant

24

11

a month ago

When I turned 30, I had been unemployed for more than a year after being laid off from my tech job.

I was getting interviews, but not offers. After enough rejection, I started wondering if I was really as good an engineer and data scientist as I thought I was.

I thought about projects where I hadn't performed as well as I wanted—and what I'd wished I'd had when I started in data science: someone showing me not just how to build these systems, but what could go wrong when they met the real world.

So in 2021, I started a blog breaking down failed machine learning and AI projects.

After a few posts, I stopped.

I didn't know how to consistently find incidents. I wasn't sure people cared. And when I found well-known examples, I wondered what I could add that hadn't already been said.

Looking back, I think I gave up too soon. I was onto something. I just couldn't see how to structure it yet.

Then AI exploded into the mainstream.

As companies put AI into customer service, healthcare, finance, legal systems, education, and more, the failures became more visible too.

Five years later, while sharpening my AI consulting practice, I found myself returning to the same question:

What can these failures teach us?

Except this time, I started noticing patterns.

Different companies, industries, and technologies could experience surprisingly similar underlying failures.

Now I'm exploring whether those patterns can become principles for diagnosing and building AI systems that behave more reliably in the real world.

I've found early signals that there may be a need for this expertise. But I still don't know exactly where it will lead.

Five years ago, that uncertainty convinced me to stop.

This time, not knowing is giving me a reason to keep investigating.

I explore what goes wrong so more people can build things that go right.

#storytellingcontentchallenge #ai #artificialintelligence #aiforensics #bytesolid by @daryleserrant

21

12

a month ago

By 36, I thought I'd be the one sending money to my mom. Recently, she offered to send money to me.

For years, I had a particular picture of what my life in tech would eventually look like.

I'd be making well over six figures, have my own place, have plenty of disposable income, and be financially comfortable enough to support my mom if the need arises.

My life at 36 doesn't look like that.

Recently, my mom called and told me she wanted to send me some money to help me.

My immediate reaction was, "You don't have to do that." Part of me didn't want to accept it. This wasn't how I thought things were supposed to work by now. I was supposed to be helping her.

But after sitting with it, I started seeing the situation differently. I probably could have managed without her help. But she wasn't being forced to give me anything. She wanted to.

This was one of the ways my mom chose to support me and express her love. I realized that by refusing someone's contribution simply because it didn't arrive in the form I wanted, I might also be denying them the opportunity to give.

I've spent a lot of my life associating success with becoming the person who has enough to give. I still want to reach a point where I'm able to do more for her financially. But I'm beginning to understand that receiving matters too.

Accepting someone's support doesn't diminish my ability to eventually give back. Sometimes accepting love means allowing someone to give it in the way they know how.

#storytellingcontentchallenge #expectationvsreality #family #motherandson #thirties by @daryleserrant

43

27

a month ago

Would I do everything the same way again? No.

There are financial decisions I would handle differently. There are risks I wish I'd taken sooner. There are also opportunities I could have pursued more effectively if I knew then what I know now.

But if anyone were to ask me if I regret taking the chance in the first place, my answer would be no.

At age 23, I was trying to figure out what I wanted from my career. I had a stable software engineering job at the time, but I was curious about doing something different.

Over the years, I changed careers, moved across the country, invested heavily in education and personal development, got laid off, worked with several companies, became involved in AI research, and eventually started to build a consulting business of my own.

That last part is still a work in progress. There are times when opportunities disappear and conversations with potential clients go nowhere. At times, I would wonder if I'm in over my head.

Recently I came across an old YouTube video I recorded when I was 23. In it, I was talking about what I wanted to do with my career. I wanted to work on cutting-edge research and I wanted to become a consultant who could help different companies build technology.

I'd completely forgotten I said that.

I realized that I'm living parts of that life right now. Watching that video made me realize how easy it is to measure yourself only by the distance you still have left to travel. Although I haven't fully realized that vision, the pursuit itself changed me.

I'm more comfortable with uncertainty. I'm more willing to take action without having everything figured out. And, perhaps most importantly, I've developed more depth as a person.

I don't know exactly how this story ends. But I'm proud that I gave myself the chance to find out.

#storytellingcontentchallenge #career #softwareengineering #dreams by @daryleserrant

54

10

a month ago

Before I was laid off, I had spent a lot of time doing meditation and other forms of inner work.

One thing the practice was helping me learn was how to experience uncomfortable emotions without immediately allowing them to control how I interpreted what was happening.

I got an unexpected call from my boss while I was in Seattle for a leadership conference that was taking place during the weekend. Before she even told me what was happening, I had a strong feeling that I was about to lose my job. I felt really anxious.

But at some point during the Uber ride from the airport, I experienced a sudden flash of calm. And at that moment I thought, "I don't think I have a job anymore, and that's okay."

I later found out that I was right. The year that followed wasn't easy. I struggled to find full-time roles and there were plenty of moments of uncertainty.

What my meditation practice gave me was more space between what I was feeling and what I allowed that feeling to mean. I could feel uncertain and still have some underlying trust that I'd eventually find a way forward.

I'm still learning how to do this, but I've started to realize that groundedness isn't about avoiding discomfort. It's about allowing space for it without letting it decide the story for you.

#storytellingcontentchallenge #layoffs #mindfullness #meditation #career by @daryleserrant

24

17

a month ago

For a long time, I had a tendency to turn outcomes into judgements about myself.

If someone else got an opportunity I wanted, maybe they were better than me.

If someone criticized something I created, maybe I wasn't as capable as I thought.

The opposite seemed true as well.

If I got the job, received praise, or accomplished something impressive, it was evidence that I really was good enough.

Herein lies the dilemma: I had experiences where I got the outcome I wanted and still wondered if I was good enough. There was always another accomplishment to pursue. Another opportunity, another person's approval that could finally provide the proof I was looking for.

Recently I've been revisiting a distinction that I learned months ago from a teacher named Masati. Judgment and validation are more like signals from a GPS.

Judgement can tell me that something isn't working, where I may need to improve, or that I'm moving in the wrong direction. Validation can tell me that something is working or that I may be moving in the right direction.

Both can contain useful information, but I've spent a lot of my life asking those signals if I was good enough. I'm beginning to realize that this is a question that they were never capable of answering.

A rejection can still hurt. Criticism can still disappoint me, especially when I've poured a lot of myself into something. Positive feedback can still feel amazing. I'm not trying to be indifferent to any of it. I'm learning to separate the information from the identity I attach to it.

Maybe I really do need to get better at something. Someone else really could have been a better fit. And maybe something I created didn't meet someone else's expectations.

That's information I can use. None of it has to become a verdict on my worth.

#storytellingcontentchallenge #success #career #growthmindset #identity by @daryleserrant

38

14

a month ago

One of the hardest software failures I've ever diagnosed was intermittent blue screen of death (BSOD) crashes on systems integration test equipment I was responsible for upgrading.

Sometimes the crashes would manifest within an hour of operation. Sometimes several hours later. And sometimes it would operate for days with no issue.

I spent months trying to figure it out. I consulted with other engineers on my team for ideas. We reached out to a hardware vendor for assistance. I also tried logging the failure, but the crash itself made collecting useful information difficult.

Eventually I came across a debugging principle that changed my approach: If a failure is intermittent, find a way to make it fail on demand.

So instead of continuing to ask, "What's causing this?" I started asking, "What conditions make this happen?"

That allowed me to isolate pieces of the system and eliminate possibilities until I eventually built a small computer program that reproduced the failure reliably.

The problem wasn't my application code. It was a faulty hardware driver from a vendor.

I've encountered versions of that lesson repeatedly since then, whether debugging software, deploying systems, or working with machine-learning models.

Sometimes the answer is buried in a log you haven't looked at closely enough.
Sometimes your code works locally but behaves differently in another environment.
Sometimes an AI model's aggregate performance hides specific classes where it's struggling.
And sometimes you have to read source code from open-source libraries you didn't write because the documentation doesn't tell you enough.

The lesson I've learned from all of this is to resist becoming attached to your first explanation.

When an AI feature underperforms, it's easy to immediately blame the AI model. The model, however, is only one component of a larger system of data, software, users, workflows, inputs, and outputs.

Before deciding what to fix, first understand what is actually failing. Every possibility you eliminate can teach you something, even when you haven't found the fix yet.

#storytellingcontentchallenge #ai #softwareengineering #ml by @daryleserrant

65

8

a month ago