Sadly, software cannot be estimated that way, and we need to stop pretending otherwise. It's a myth—a pervasive one—and perpetuating that myth only perpetuates its harm.

Sure, if a task is almost identical to some previous task, a precise and reliable estimate really can be communicated. Unfortunately, that almost never happens. If the work ahead were so similar, someone would have done it already using copy and paste. If it's similar and the engineering team wanted to set themselves up for success later, they'd refactor the system. That refactoring itself can be a hazy fog of unknowns. We've all been there many times.

In my fifteen years as a software engineer, I've worked in many different kinds of organizations, from a small non-profit research and development lab (the Open Publishing Lab at RIT), to a medium-sized, quasi-non-profit business with a deep and pervasive developer culture (the Mozilla Corporation), to a for-profit startup in the music industry (Inveniem), to a publicly-traded conglomerate with an estimated 100,000 employees (Honeywell). Although I have fond memories with all of my former employers, in my experience, not one was even remotely better or worse than any other at software estimation. Clearly, something more fundamental is wrong.

I think the problem is obvious. They all wanted to know the unknowable.

The problem

I once worked with someone who liked to compare software development to parcel delivery: communicate a date when the package is expected to arrive. It's okay if there needs to be a delay, as long as that delay is communicated clearly. While the flexibility with delays was genuinely appreciated, the broader analogy soured on me over time. In truth, estimating software is nothing like estimating package delivery. The former involves doing something that has never been done before, while the latter is completed billions of times each year. Does Amazon have an advantage when forming estimates? You'd better believe it.

Software development is one of the only professions where we are asked to accurately state how long it will take us to do something that has never been done before in the history of the universe.

—Unknown, from a screenshot on X

How long will it take for Amazon to deliver a board game to Zara in Cincinnati? Well, probably about as long as it took last time. Yes, every delivery is different, but there must be millions of deliveries of similar items to nearby locations with similar weather forecasts and other pertinent factors being similar or identical. How long will it take? Probably that long.

By contrast, how much time will be required for me to change code I've never seen? I truly have no idea. How about adding, say, karaoke support to a streaming music app, one with millions of lines of code I've never read, where myriad caveats exist for different software clients, different spoken languages, and different copyright laws. How long will that take? Except in extremely broad terms, your guess is as good as mine. It will take more than one week, and it can probably be completed in less than one year. Beyond that, there's not much more that can be promised, at least in the beginning. More on that in the next section.

No, estimating software development is nothing like estimating like parcel delivery. It's more like estimating the pace of scientific progress. We simply don't know what we don't know. We can't know until we do it.

As much as possible, organizations should communicate that software will be ready when it’s ready. If some software must be released by a certain date, the feature list should not be guaranteed. I know this is easier said than done, believe me, but reality is what it is. Software simply cannot be estimated the way organizations imagine. Some higher-ups, especially those who have not been software engineers, may find this outrageous. They may demand exact estimates regardless. To them, I would say, “Fine, but let’s be honest, how’s that working for ya? How often are you given estimates that actually turn out to be correct?”

There's a reason you're reading this blog post. Something isn't working, and it's not about discipline. It's not about communication. It's not about skill. You're on the wrong path. Skeletons and vultures surround you. It's time to turn around.

Thankfully, there is a better way. Your manager may not like it at first, but over time, they may come to see that estimates of this type are actually useful, unlike the aspirational and unreliable estimates they're accustomed to.

The solution

To explain this kind of estimation, I need to define two terms. First, I'm using the term accurate to describe any estimate that is correct within a given range of dates. Defined this way, even a very broad estimate can be accurate. For example, an estimate that a feature will be completed some time between March 1^st and December 31^st would be accurate, as long as it turns out to be true. Precision is another matter. Precision refers to how narrow a range of dates is, with more narrow ranges being more precise. The previous feature estimate isn't very precise, but it would be more precise if it predicted a completion date between, say, April 1^st and July 1^st. This is how Rapid Development defines the terms, and I think it’s a good approach. In fact, most of this advice is lifted from Rapid Development. I recommend reading it. It's a tome, so focus on the sections that are most pertinent to your needs.

Crucially, estimates can and should become more precise as time goes on and details become more clear. If a team begins with an estimate of some time in Q1 or Q2, but the feature is more difficult than expected and they realize they need to revisit the database schema, perhaps the estimate could be narrowed to some time between May 1^st and June 15^th. If the team is nearing the end and the final code reviews go more smoothly than expected, maybe the estimate could narrow to some time between May 16^th and May 31^st. If a critical issue is found elsewhere in the codebase, taking the team off the feature for a while, maybe May 29^th becomes the most likely date of completion. The point is, there's practically no way to predict those circumstances in advance, but they can inform better estimates over time, as long as the estimation method supports that.

Did you instead randomly pick a date three months in advance? Too bad. You're screwed.

Reality-driven estimation

So how should we estimate? Strive for accuracy, start with low precision, and seek greater precision over time. As another example, you might consider communicating a project end date some time between Q2 and Q4, then later refine that to some time in the fall, then some time in October. Finally, at the very end of the project, you might be able to promise a specific week or even a specific day. That is an approach to estimation which acknowledges reality. That is an approach to estimation which acknowledges that surprises happen.

Do organizations want to hear this? Unfortunately, no, I don't think so. It's a tough sell. Organizations want certainty, and they want it now. Good luck telling your boss that you don’t know exactly how much time and money you're going to spend. Even the project management apps we use typically require that an estimate be recorded as a specific date. Talk about encouraging bad practices.

Pushing back is hard, and as someone who was raised to never question authority, even when that authority is obviously wrong, I happen to be particularly bad at it. That's too bad. Our bosses pay the bills, but we do the work. It's time we start saying, “The work can't be done that way, and if I were to tell you otherwise, I'd be lying to you.”

Reality is what it is, and the best we can do is provide estimates with gradually increasing precision. Organizations can pretend otherwise, and countless surely will, but again, I would ask, “how's that going?” Organizations ignore these facts to their own detriment, at the cost of time, money, and frustration. I wish them luck.

For everyone else, consider embracing reality. It's not perfect, but it's the best we can do.

#Life #Quotes #SoftwareDevelopment #Tech

There was an article where GPT-4 attempted to write three posts in my voice, but it was clearly explained as such. I also occasionally use AI to help with my grammar or other phrasing, but in those cases, I never lift more than a few words from the response. I consult with AI about wording like that about as often as I consult a thesaurus.

AI slop doesn't worry me as much as it worries some others. I expect AI-generated content will improve dramatically over time and will become indistinguishable from content produced by the human mind. We may already be there, for all intents and purposes. As far as I'm concerned, that's not the point. Again, the process of writing benefits me, and I'm not willing to forego that benefit.

There is an upcoming post which uses an image generated by ChatGPT. That post credits ChatGPT as the creator, and I plan to always credit AI for images it creates.

#Life #Tech

—P.J. O'Rourke

I love this quote, because I've been that guy. I've been the guy who thinks he can save the world but who literally doesn't help his mom with the dishes when he visits. Thankfully, I've dramatically softened in my activism (appropriately discussed ever so briefly in a recipe but nowhere else on this blog), if it can even be called activism any more, and I did help my mom with the dishes during my most recent visit, although I'm sure I could have done more.

I interpret the statement like this: riding a “high horse” can be fun, and there really are important societal problems that ordinary people can help improve. That said, there are always more ordinary problems that need attention, and sometimes fixing those things goes further than protesting in the streets.

#Life #Quotes #SocialMedia #Tech

Some people are motivated by money, and that's completely reasonable. It pays the bills! It's just not who I am. It's not who I've ever been. Money, metrics, status: I care about those things like penguins care about Pilates. I'd rather stare at a wall. (I wrote that last sentence before realizing what I was saying, but it's absolutely true, in more ways than one. If you get the reference, send me an email.)

Don't get me wrong. I can be deeply motivated under the right circumstances. You can hardly pull me away from the computer when I'm learning, iterating, honing my craft, and producing something I'm proud of. That's where I find flow. “Faster, faster, faster, more, more, more!” just because that’s what your boss wants? No, that doesn't work on me.

I'm amazed that style of management works on anyone, to be honest, but it must. I suppose some people who are motivated by promotions and prestige can clench their teeth and bear it. Maybe they even enjoy the challenge. Me? I don't see the point. Life is short, and nobody spends their final moments reminiscing about their corner office or their fancy car. Let's be honest, those things lost their luster after one week.

I regret not being more clear about this aspect of my personality in the past. Moving forward, I want to embrace who I am. If others don't like it, that's fine, but they're probably not the right people for me, and I'm probably not the right person for them.

#Favorites #Life #Maxims #SoftwareDevelopment #Tech

The next day, someone in an online class expressed confusion about how to meet their coach for a scheduled appointment, but the coach had no appointment on their calendar. Similarly, years earlier, a relative told me they had ordered an Uber, but it never showed up.

In all of these cases, I'm almost certain the users did not click the final button labeled Order, Confirm, Submit, or similar. (The restaurant owner blamed browser cookies, but that has nothing to do with it.)

I can hardly blame these users, though. I once showed up to a hotel, thinking I had booked a room with them, only to find that they had no reservation under my name. I looked for the confirmation email, and sure enough, it didn't exist! I needed to quickly book with some rinky-dink place across the street. I had made the same mistake!

This isn't about tech literacy or intelligence; it's about bad user interface design. A depressing amount of software ignores the basic rules of usable design. In this case, what looks like a confirmation screen (i.e., “your order has been placed”) may actually be a confirm screen (i.e., “tell us that you're sure”). When users are on those screens, their loci of attention may be on the order details rather than the user interface. They want to know that the flight time is correct. They want to know that the price is correct. They may not be paying attention to the Confirm button, which is probably offscreen anyway. They may not be looking at the tiny text at the top which reads, “Please review this information before booking your flight.” They are focused on something else, which may explain why they miss context clues in the periphery. Jef Raskin discusses this at length in his book, The Humane Interface.

So before leaving an app, website, or other ordering system, be sure to confirm, confirm, and confirm again. Slow down. Read carefully. Scroll. Zoom in and out. Make sure there's nothing else you need to do. A few seconds of prudence may prevent lots of frustration later.

#Life #SoftwareDevelopment #Tech #TechTips

A while back, I watched a video about all the ways hotels are trying to save money by, among other things, eliminating storage space, removing desks, pressuring guests to work at the bar, where they can make more money, and making the in-room bathroom less private. These changes are, of course, driven by metrics like “GSS” and “ITR,” whatever the f@*k those are.

Is there a kernel of truth to all of this? Sure. Aloft Hotels are cozy, and they seem to follow this playbook. I didn't mind staying in one when I was stuck in San Francisco for one night more than ten years ago. Would I want to stay in one of their rooms during a business trip or anything else lasting more than a couple of days? Hell no. I'd like a desk and somewhere to put my clothes. (I know, I'm so needy. I travel with clothes.)

Metrics are fine, sometimes, when their use is limited and their shortcomings are genuinely appreciated. Taking them too seriously and letting them make the decisions, however, is a recipe for disaster. Hard questions demand more thoughtfulness than that. “GSS” and “ITR” are meaningful until they aren't, and nobody is going to find solace in those abbreviations when generations of potential customers steer clear of your business because they actually want something good.

Sadly, I don't think most businesses think that far ahead.

Show me the metric which proves that your business isn't taking a massive risk by ignoring common sense. Until then, I don't care about “the numbers.”

#Life #SoftwareDevelopment #Tech

Say you're into politics. [TikTok] will push you further and further into the extreme edges of whatever side you're on. Why? Because outrage and confirmation bias keeps you glued to your screen! This isn't just bad for your worldview, it's bad for society. Echo chambers breed division. They make people more certain they're right, more hostile to differing opinions, and less likely to engage in actual conversation.

—If TikTok Were Honest

Let's not gloss over the reminder that this is bad for your worldview. That's one of my biggest problems with social media. In making people more extreme and less aware of differing opinions, their persuasive ability weakens and they become counterproductive in their activism. To add insult to injury, they sometimes actually come to think of themselves as the sacred protectors of their various causes. Give me a break.

Don't leave social media because I told you to. Leave social media because it's making you hurt the people you're trying to help.

#Life #Maxims #SocialMedia #Tech

I'm surprised, disappointed, and a bit embarrassed I didn't know about it until now.

#SoftwareDevelopment #Tech

I don't want to be too cynical, but using social media for this purpose does seem like opt-in advertising on some level. It's too bad that many of these users will also be manipulated into liking, commenting, buying, sharing, following, radicalizing, and you know, dismantling democracy.

#Life #SocialMedia #Tech

At some point, it apparently became fashionable to slap the label narcissist on anyone who has behaved badly, as well as many people who haven't. Someone's ex is a narcissist. That one's boss is a narcissist. Everyone's parents are narcissists. What in the world is inspiring people to talk like this? Narcissistic Personality Disorder (NPD) does exist, yes, and there may be some very conspicuous examples of it in public life, although I'm not qualified to diagnose anyone. Still, it's a minority disorder. The Cleveland Clinic reports that NPD affects around 0.5% to 5% of Americans. Clearly, most people who behave badly from time to time do not qualify for a diagnosis. Moreover, mental illnesses like anxiety and depression are far more common.

Yes, sometimes people treat others badly because they are narcissists, but others are unkind due to their depression, anxiety, obsessive-compulsive disorder, bipolar disorder, borderline personality disorder, addiction, or one of the dozens of other psychological afflictions that cause so much pain. In most cases—not all, but most—I'm sure those suffering with these ailments endure much more agony than the people around them. Of course, that's assuming the target of the “narcissist” label is even clinically unwell. Maybe your boss just wants to further their own goals at the exclusion of yours. That's not narcissism. That's not mental illness. That's just corporate life. (I would argue that any manager pursuing their own goals at the exclusion of yours is a bad manager, but that doesn't make them a narcissist.)

Language evolves, and I suppose people can use the term narcissist to mean brute, if they choose. The dictionary wasn't handed down from the heavens, unchangeable. I just worry that being so sloppy with terminology unfairly demonizes the vast majority of mental illnesses that inspire unusual behavior for other reasons. I also think it can suggest a degree of intent that simply doesn't exist. Maybe that person at the convention shouted at you because they struggle with anger or because they never learned how disagree respectfully, not because they want to feel superior to you.

I do wonder—and this is pretty speculative—whether some people are so cavalier with the term narcissist because they want to deflect attention away from their own narcissism of a different kind. I'm not talking about clinical narcissism, the type that seriously harms oneself and others, but rather more ordinary narcissism, the kind that leads one to believe that anyone actually cares about their status updates. I think it's plausible that social media does foster some amount of casual, everyday narcissism. Could it be that people throw the term around because they're uncomfortable facing their own shrouded narcissism of a different kind?

Instead of throwing labels around, maybe we should spend more time looking in the mirror—in a healthy way. I will try to do the same.

#Life #SocialMedia #Tech