Estimating Time

This document explores realistic time estimation for IT projects, addressing human tendencies toward excessive optimism by using comparison with similar past tasks, breaking large projects into estimable pieces, accounting for integration overhead and unforeseen obstacles, applying experience-based multipliers, and documenting actual versus estimated times to continuously improve estimation accuracy through retrospective analysis.

The Automation Decision Framework

Cost-Benefit Analysis Basics

As called out before, when deciding whether a manual task needs to be automated, two things should be considered: how many times the task will be done over a period of time and how long it takes to do it manually.

Automation decision factors:

Example automation decision:

From this calculation, figuring out if that number is larger than the time it will take to write the automation becomes possible. This sounds great in theory, but the problem is that the time it will take to write the automation isn’t known until it’s actually written.

The Estimation Challenge

The Unknown Variable Problem

All that can be done is estimate it, and most people are really bad at estimating how long tasks will take.

Why humans are bad at estimation:

Overly Optimistic Instincts

Typically, the first instinct is to consider how much can get done with an ideal amount of focus on the work and a full grasp of the problem being solved.

Ideal conditions vs. reality:

Forgotten Obstacles

Taking into account the many obstacles that might be faced is forgotten, including finding a bug that’s unknown how to fix, being interrupted by a problem needing more urgent attention, or discovering that a new tool doesn’t work well with the rest of the tools in place.

Common forgotten obstacles:

Warning

The biggest estimation mistake is assuming ideal conditions with full focus, complete understanding, and no obstacles. Every project encounters bugs, interruptions, tool issues, and unexpected complexity that must be factored into realistic estimates.

Realistic Estimation Principles

Avoiding Optimistic Bias

So if trying to estimate how long it will take to complete a project, big or small, being realistic is necessary. Avoiding being overly optimistic with time estimates is critical.

Estimation principles:

Comparison-Based Estimation

The best way to do this is to compare the task being tried with similar tasks that have been done before. This way, estimation is not based on how long the project would be liked to take but on how long other similar projects actually took in the past.

Comparison-based estimation process:

Types of similar tasks to compare:

Task Decomposition Strategy

Breaking Down Large Tasks

If the task at hand is large, it might be hard to find something similar enough to use for comparison. So to make a better estimate of a bigger than average project, chopping it up is the solution.

Task decomposition process:

Example: User authentication system decomposition:

When to Split Further

Split the task into smaller steps. Compare each step to a similar task that’s been done in the past and assign an estimated amount of time to each step based on that. If one smaller step is still too large, then split it into even smaller pieces until each piece of the puzzle can be compared with something that’s been done before.

Splitting criteria:

Integration Overhead

Adding Assembly Time

Once all those estimated times are obtained, just adding them up will give a rough estimate of how long the whole task will take. But even that’s going to be optimistic since putting all the pieces together will take additional time.

Integration overhead factors:

So once having a rough estimate of the total time of all the steps, factoring in some extra time for integration is wanted. This should also come from prior experience.

Integration multipliers by project type:

Think about how long it took to integrate the pieces of a project before, and a rough idea of how much to add to the previous value will emerge.

Note

Integration time is often the most underestimated part of projects. Components that work perfectly in isolation frequently encounter issues when connected: timing problems, data format mismatches, unexpected interactions, and emergent complexity that wasn’t visible in individual pieces.

Experience-Based Multipliers

Accounting for the Unknown

Knowing how optimistic humans are, even after basing estimations on previous experience, the number arrived at is going to be very close to the best possible scenario.

Optimism in estimation stages:

Even if prepared for something to go wrong, it’s impossible to anticipate new unknown bumps in the road, so taking this estimation and multiplying it by a factor is necessary.

Determining the Multiplier

Once again, this factor works best when it’s based on previous experience. So if the last time this exercise was done, it took three times longer to complete the task than planned, go ahead and multiply the estimation by three.

Calculating personal multiplier from history:

Multiplier guidelines by experience level:

Multiplier adjustments by project characteristics:

Why Multipliers Aren’t Inflating

This might seem like inflating the numbers, but remember, having a grounded estimate of how long it will take to complete the task is the goal. That means taking into account the obstacles that will certainly be run into but haven’t been come across yet.

What multipliers account for:

Estimation Granularity

Level of Detail

No matter how detailed the estimation is, the final estimation won’t ever exactly match the time it takes, but it will give a rough idea of whether the task can be completed in a few hours, days, weeks, or months.

Estimation accuracy by time frame:

Appropriate granularity by time frame:

Important

Don’t over-precision estimates. Saying “4.3 days” or “37 hours” implies false accuracy. Use appropriate granularity for the time frame: hours for same-day work, days for week-long tasks, weeks for month-long projects. The further out the estimate, the less precise it should be.

Documentation and Learning

Recording Estimates

Once the estimation has been made, noting it down somewhere so that checking later to see how close it was to the original number is possible. Future estimations can be adjusted based on that.

What to document:

Estimation tracking template:

Continuous Improvement

Estimation improvement cycle:

Metrics to track over time:

Stakeholder Communication

Setting Expectations

Wanting to communicate with those affected and let them know when they can expect the task to be done is also important.

Communication elements:

Communication timing:

Communication by stakeholder type:

Caution

Never commit to a single date without acknowledging uncertainty. Use ranges (“10-15 days”), confidence levels (“likely by Tuesday”), or regular updates (“will reassess Friday”) to avoid over-committing to optimistic estimates that damage credibility when missed.

Conclusion

Estimating time for IT projects and automation tasks requires overcoming natural human optimism bias that focuses on ideal conditions with perfect focus and full understanding while forgetting inevitable obstacles like unknown bugs, interruptions, tool incompatibilities, and scope discoveries that add 50-300% to initial instincts. The most effective estimation approach uses comparison with similar past tasks to base estimates on actual historical experience rather than wishful thinking, asking “how long did the OAuth integration really take?” instead of “how long should it take?”, providing evidence-based reality checks against optimistic instincts. Large projects require task decomposition by splitting work into smaller comparable pieces, estimating each sub-task against similar past work, and recursively subdividing until each piece matches something previously done, then summing individual estimates to get a baseline total that still needs adjustment. Integration overhead accounts for the additional time required to connect components that work individually but encounter issues when combined, adding 10-70% depending on project complexity based on prior experience with similar integration challenges, covering debugging interactions, data flow testing, configuration management, and interface alignment activities often invisible during decomposition. Experience-based multipliers address unavoidable unknowns by applying factors of 1.5-4.0× depending on individual accuracy history and project characteristics, with junior developers using 3-4× and experts using 1.5-2× adjusted upward for new technologies or vague requirements, covering undiscovered bugs affecting 80% of projects, requirement changes in 50-70%, tool issues in 30-50%, and other obstacles impossible to anticipate but certain to occur. Estimation granularity should match time horizons with hour-level precision for same-day tasks but week or month-level precision for longer projects, avoiding false precision like “4.3 days” that implies impossible accuracy, aiming instead to determine whether work takes hours, days, weeks, or months rather than exact durations. Documentation and continuous improvement require recording original estimates with assumptions, tracking actual time spent, calculating accuracy ratios, analyzing what was missed, adjusting multipliers and approaches based on evidence, and sharing learnings with teams through retrospectives, building personal and organizational estimation capability over time through systematic learning from prediction versus reality comparisons. Stakeholder communication must set realistic expectations through ranges rather than single dates (“10-15 days”), explicit confidence levels and assumptions, identified risks that could extend timelines, milestone-based progress tracking, and regular updates at 25%, 50%, 75% completion that revise estimates based on actual progress and newly discovered complexity rather than maintaining optimistic initial commitments that damage credibility when inevitably missed.

FAQ