Thoughts on Software Development Estimates and Feature Creep.

I just finished working on an app and I couldn’t imagine facing certain types of problems at a medium to large company. When joining enginneering teams we sometimes have in mind that the things we encounter as a freelance developers doesn’t happen as companies have more resources to be structured in the organization of a project. After a while you come to realize those companies are made up of people who are no more or less better than you in insight, talents, organizational skills and determination.

I was tasked to work on extending an app that has already been built where some functionalities needed to be added and polished; but with increasing time and number of meetings, bugs were found, and when those bugs were found the clients always had a subtle “suggestion” to add so and so “little” functionality and/or modification. Being with a senior dev on the project(the one who actually worked on the first version) I thought he would be the one to push back. Oh boy was I wrong; everything meeting came with a “bug” where sometimes the bug was the fact that the client didn’t like what they suggested before so we had to roll back sometimes to add new functionalites. Every time those concerns were brought up, the senior will be like “no problem” and I will be like 😟. In my mind I was like we have a deadline. With this attitude we won’t even have an actual working version by the end of it. It dragged for a whole couple of months before we arrived at something that was “good enough”.

Generally software development estimates are often way off because many aspects of software development are impossible to estimate. If you have machine learning models built in-house that drive the application it can become a nightmare if proper time wasn’t allocated.

Here are some reasons software development estimates fail;

1. “Done” is variable. Aspects of acceptable quality like performance, code quality, security, accessibility, reusability, readability, and usability are hard to specify and quantify. This leads to time-consuming arguments and negotiations over when code is truly done. A good software spec document where both parties have agreed on the completion of the work can help in this scenario.
2. Dev environment issues are unpredictable. Examples include hardware issues, framework and library bugs, slow servers, service interruptions, internet problems, access control issues, third party outages, VPN issues. Some companies have entire libraries to do stuff their own way. It can really cause unpredictable issues when working with external clients.
3. Bad requirements are hard to detect early. Incomplete, vague, conflicting, outdated, or incorrect requirements are often hard to detect until implementation, deploy, or usability testing. Requirements are “lossy” but being able to be as detailed as possible given the time and resourcesfrom the onset helps in reducing these issues drastically.
4. Developer velocity varies daily. Developer effectiveness, and efficiency varies widely. It depends upon tech expertise, existing code quality, domain knowledge, competing priorities, dev environment stability, turnover, domain knowledge, sickness, time off, and more.
5. Communication overhead is dynamic and unpredictable. Overhead varies based on team size, solution complexity, documentation needs, turnover, coupling, and approach. Every extra human adds overhead that’s hard to quantify.
6. Cross-team dependencies reduce control and increase the risk of delays. Cross-team projects require multiple teams to deliver on time. That’s hard to do given all the previous variables listed above.
7. Untestable code is hard to identify up front. If a new feature interacts with code that isn’t friendly to testing, refactoring the code may be required. This is hard to detect up front when estimating effort.

In summary all I can say is a good requirements doc and good documentation on an already existing project goes a long way to help in a project that will be passed from engineer to engineer. It seems like in some companies, these are a luxury.

References

[1] Eric Bernhardsson, Why software projects take longer than you think: a statistical model(2019), Erik Bernhardsson’s Blog