Computer Programming/Coding Style/Minimize nesting

Deeply nested code is a common feature of structured programming. While it has some advantages, discussed in that section, it is frequently considered hard to read and an anti-pattern: “Flat is better than nested”.[1]

Specifically, nested control flow – conditional blocks (if) or loops (for, while) – is hard to understand beyond three levels of nesting,[2][3] and has high cyclomatic complexity. This is known as “Dangerously Deep Nesting”[3] or, in the case of nested if statements, the “Arrow Anti Pattern”, due to the following shape:

ififififdo something
endif
endif
endif
endif

This has a number of problems:

The code is hard to read.

Context is hard to understand, due to multiple levels of indentation.

Cleanup happens vertically far from the original cause: if a resource is acquired (say, memory allocated, file opened) at the top, in one indentation level, the cleanup occurs at the same indentation level, but at the bottom, vertically far.

Other than refactoring or avoiding this code, one technique to handle deeply nested code is code folding in editors – this allows you to collapse a block, yielding abstraction and allowing you to see the surrounding code easily without the intervening code (so resource acquisition and cleanup are both visible).

If several if clauses are just tests (without any intervening code), these can be combined into a single test. Compare:

ifa:ifb:...

to:

ifaandb:...

Inline function calls with boolean short-circuiting

If the only body of the if clause is a function call and assignment to perform a test, followed by another if clause, in language such as C where assignments are expressions (have a value) and boolean expressions are short-circuited, these can be combined:

if(a){intb=f();if(b){...}}

if(a&&intb=f()){...}

Auxiliary variables or functions

Auxiliary variables are useful when a complex expression is inlined in code, notably a boolean expression or anonymous functions.[5] Using an auxiliary expression both reduces the nesting, since it is no longer included within another expression, and the variable name documents the meaning of the expression. For complex boolean expressions, another alternative is a separate function which is called, rather than an auxiliary variable.

The most significant solution is early return, which has several forms, notably a guard clause.[4] Avoiding nested control flow is a fundamental reason for non-local control, notably: return (value), raise (exception), continue, and break. A common pattern is to replace an if-then or nested if ifs by if not/return-continue (return/raise if a function, continue/break if a loop body).

Compare:

ifa:...ifb:......

to:

ifnota:return...ifnotb:return......

Similarly, compare:

foriinl:ifa:...ifb:......

to:

foriinl:ifnota:continue...ifnotb:continue......

This reduces the nesting and makes the flow more linear – either go further down the block, or return/continue.

This pattern is called a “guard clause” when the checks appear at the start of the code and check preconditions. However, it is used more generally to finish processing and return immediately once work is complete or a value has been computed:[3]

“Use a return when it enhances readability: In certain routines, once you know the answer, you want to return it to the calling routine immediately.”

However, early returns are potentially confusing and error-prone, notably due to the issues of cleanup, and go against a central tenant of structured programming, namely a single exit point per routine.[3]

“Minimize the number of returns in each routine: It’s harder to understand a routine when, reading it at the bottom, you’re unaware of the possibility that it returned somewhere above. For that reason, use returns judiciously–only when they improve readability.”

In the absence of cleanup – when a function is just computing a value or producing side effects – early returns have fewer potential problems. In the presence of cleanup, some languages have facilities that facilitate cleanup even with returns (such as “finally” clauses, “atexit” in Unix or Python, or “defer” in Go). Another option is to keep a single exit point at the end, following a cleanup clause, and instead of early returns, jump (goto) the cleanup clause.

In cases of complex nesting – nested if/then/else statements or multiple if statements at a given level – often the logic is simply multiple exclusive conditions, which can be handled by testing for each condition in turn, executing code if relevant and then returning or using an elif, allowing a flat structure and making the complete condition clear.

A subtler issue occurs when processing a sequence of data by first doing one thing on some of the data, and then switching to a different state and process the rest of the data. One can do this by nested loops, but more natural is to break the loop and then continue in a separate loop.

In many languages, such as C, this is done by having an auxiliary index variable that is shared between the two loops.

A more complex example occurs when you want to switch back and forth between two ways of processing data, such as iterating through a string overall vs. within words. In general the most elegant solution is via mutually recursive coroutines (with tail calls), operating on a shared iterator (or index variable), though in languages without coroutines this is instead done via a state machine, or sometimes mutually recursive subroutines.

In simpler cases where there is a main loop and a secondary loop (such as looping through a string, secondarily operating on its words), there is a natural nested structure. In this case simply factoring the secondary loop into a separate function is sufficient to remove the nesting. Compare:

While deeply nested code within a single function is undesirable, having separate modules, functions, and nested functions is an important form of modularity, particularly due to restricting scope. As a rule, it’s better to have all functions in a module be related to each other (for cohesion), which favors a high level of factoring and separate modules. That said, this can increase module complexity, particularly in extreme cases such as Java, with its restriction of one public top-level class per file.

Helper functions used only in one place are often nested within the function that calls them, in languages that support nested functions – this gives them access to the enclosing scope (reducing the need to pass parameters back and forth), and restricts their own scope.

However, in the case of long helper functions, it is generally clearer to have a separate, private, top-level function, particularly if they do not need access to the state of the containing function – nested functions are primarily useful for having access to the enclosing state, not because their own scope is restricted. This shortens the main function, and reduces the indentation and complexity of the helper function, because the state of the enclosing function is no longer accessible.