KidzSearch Encyclopedia:Avoiding MediaWiki expansion depth limit
 | This essay contains the advice or opinions of one or more Wikipedia users. Essays may represent common ideas, or ideas that many users would not support. They are not rules. Think carefully about what they say before following them. |
This essay, WP:Avoiding MediaWiki expansion depth limit, covers issues about the MediaWiki version 1.16 "expansion depth limit" for the nesting of templates and if-logic. All during 2009-2010, the nesting limit has been only a mere 40 levels of nested if-then-else (or nested templates) invoked inside other templates. The severe limit is so restrictive that some templates cannot function properly when invoked inside other large templates. NOTE THE DANGER: When the depth limit is exceeded, not all templates die, but rather, they simply generate the wrong results (from the remainder of the template) and keep going, as if nothing were wrong. The technical limits of expansion depth are explained in the pages:
- m:Help:Expansion depth & m:Help:Expansion#Expansion depth limit, at Meta.wikimedia.org
The limits were chosen to thwart denial of service (DoS) attacks being caused by very large or complex templates.
What is not affected: Typically, large templates can be invoked, together, in article text without reaching the limit. The main worry is when using large templates inside an infobox or deeper inside the markup of other templates (when editing a large template). For example, the large template {Convert} can be used with {str_left} to "show" the length of 2 meters:
- If 1 ft is "x" then 2 m: {{str_left|xxxxxxxxx|{{convert|2|m|ft|0|disp=number}} →
If 1 ft is "x" then 2 m: xxxxxxx
- If 1 ft is "x" then 2 m: {{str_left|xxxxxxxxx|{{convert|2|m|ft|0|disp=number}} →
Using large templates together in an article paragraph is rarely a problem.
Reducing expansion depth: The nest-levels can be reduced, inside large templates, by rewriting some of the nested if-else-if-else logic as non-nested if-then-if-then-if-then, or using a #switch outside of the if-then logic. In some cases, avoiding the use of other templates inside a template can also reduce the nesting: whereas using a #ifexpr increases the depth by 1 level, invoking another template containing that #ifexpr increases depth by 2 levels. Putting a default value in a parameter does NOT increase the nesting level: {{{1}}} and {{{1|78.5}}} are both at zero (0) levels of nesting.
Checking the current expansion limit
The current limit (as 40 levels of nested templates) can be demonstrated by the following live examples which repeatedly nest the Template:1x to try multiple levels:
* Nest 40 templates:
{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x|'''Level 40 here'''.}}
}}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}}
}}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}
- Nest 40 templates: {{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|Level 40 here.}}
}}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}
By contrast, the following examples, with more than 40 nested templates, will cause problems, as with excessively nested templates all during 2009-2010:
- Nest 41 templates: {{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|This is level 41.}}
}}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}}
In every usage, many articles contain combinations of large templates, such as infoboxes which check string values, and those combinations can exceed the expansion depth limit.
Reducing the if-else nesting
Within a template, the nesting can be reduced by moving each if-expression to be outside another if-expression, or by combining the logic into compound conditions, such as "#ifexpr:|a=b and c=d..." rather than have a #ifexpr nested inside an outer #ifexpr. However, care must be taken to not change the overall effect of the logic, when shifting the nesting of each #ifeq or #ifexpr.
Changes can be tested faster by copying a section of code to be edited (and debugged) separately. It can be easier to copy a section of code to the top of a template during edit-preview, or to just edit an empty page, copying a section of code, for checking repeated use, during edit-preview, with different values set as the defaults for parameters.
As an example, a 3-nested stack for 3 #ifexpr can be reduced to a sequence of 3 separate uses of #ifexpr:
Original 3-nested #ifexpr:
{{#ifexpr: {{{1}}}*3 = 12.6| is 12.6
|<!--else-->{{#ifexpr: {{{1}}}*3 = 27.3| is 27.3
|<!--else-->{{#ifexpr: {{{1}}}*3 > 57| is over 57}}
}}
}}
Reduced as 3 separate #ifexpr:
{{#ifexpr: {{{1}}}*3 = 12.6| is 12.6}}{{
#ifexpr: {{{1}}}*3 = 27.3| is 27.3}}{{
#ifexpr: {{{1}}}*3 > 57| is over 57}}
Alternatively, a #switch can be used to check a parameter when equal to some specific values:
Using #switch for equal values, plus #ifexpr:
{{#switch: {{#expr: {{{1}}}*3 }}
| 12.6 = is 12.6 | 27.3 = is 27.3 | #default = <!--continue-->
}}{{#ifexpr: {{{1}}}*3 > 57| is over 57}}
By keeping the nesting of #if, #ifeq, #ifexpr or #switch to just 1-level coding, then the template would have a total nesting depth of just 2 levels. Of course, some cases will require nesting of the #ifexpr coding to handle multiple conditions which trigger extended processing with some nested #ifexpr coding to handle other options.
In general, a total expansion depth of 10 levels should be viewed as acceptable, but the specific restrictions will depend on how often a template might be used in large templates. The amount of re-writing needed, to reduce the nesting levels, will depend on the likelihood that a template will be used in combination with other templates which have large, multi-nested expansions.
In many computer systems, such nesting of if-else logic is allowed to exceed 100 levels, or perhaps unlimited levels, and hence, many people from computer backgrounds might be stunned to realize that the MediaWiki parser had severely limited such nesting to a mere, shallow 40 levels deep, all during 2009-2010.
Reducing a mathematical formula
As noted above, the parser function {{#expr:...}} consumes 1 level of expansion depth, as does {{formatnum:...}}. So, a calculation which removes commas, multiples the amounts, and then re-adds commas will nest 3 levels deep:
- {{formatnum: {{#expr: 5* {{formatnum:21,001|R}} }} }} → 105,005
Omitting formatnum reduces 1 level: In many cases, it is unavoidable to remove commas from numbers, so formatnum is then used to drop any commas, {{formatnum:21,001|R}}, but in an extreme case, require the input to contain no commas, and reduce the expansion depth by 1 level by omitting {formatnum}.
Combining nested expressions avoids 1 level: In rare cases, a calculation might contain another nested calculation. The refactoring of both calculations into a single {#expr} can reduce the nesting by 1 level:
- {{#expr: 10 * {{#expr: 2 + 3}} }} → 50
- {{#expr: 10 * (2 + 3) }} → 50
The use of parentheses "( )" allows a single {#expr} calculation to combine each formula, inside others, as only 1 level of expansion depth.
Big templates with restrictions
Among the affected templates, with many levels of expansion depth, the measurement converter, Template:Convert, has often been involved in nesting problems. Due to Convert having been designed to handle thousands of options (for converting various units) by invoking many tiny templates, the nesting for {Convert} has reached 28 of the 40 levels of nested logic. Fortunately, when rounding or precision (sigfig=3) is specified, then Convert can run within only 18 levels, allowing 22 levels of outer nesting.
Note how Convert can run when nested at 22 levels deep:
: Nest 22 templates & {Convert|5|km|mi|0}:
{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x|{{1x
|{{Convert|5|km|mi|0 }}
}}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}}
}}}}
- Nest 22 templates & {Convert|5|km|mi|0}: {{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|5|km|mi|0 }}
}}}}
However, when the nesting is increased to 23 levels, note the errors generated due to exceeding the expansion depth limit:
- Nest 23 templates & {Convert|5|km|mi|0}: {{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|5|km|mi|0 }}
}}}} }}
Incorrect results with no warning messages
The Template:Str_len, which gives the length of a character string ( {str_len|ABCDE} → 5), can return incorrect results inside a large template, when exceeding the expansion depth limit, as in the following example of a 29-character string seen as only 9 long:
: Nest 31 templates & {Str_len|123456789.123456789.123456789}:
{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x|{{1x
|{{1x |{{Str_len|123456789.123456789.123456789}}
}}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}} }}}}
}}}} }}}} }}}} }}}} }}}} }}}}
- Nest 31 templates & {Str_len|123456789.123456789.123456789}: {{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|123456789.123456789.123456789}}
}}}} }}}} }}}} }}}} }}}} }}
However, when the outer nesting is reduced, then the count (for the string length) reaches the accurate figure of 29:
- Nest 30 templates & {Str_len|123456789.123456789.123456789}: {{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|123456789.123456789.123456789}}|
}}}} }}}} }}}} }}}} }}}}
The amount of nesting, required to run a template, often depends on the particular values of the parameters, such as the following examples with a 39-long string:
- Nest 30 templates & {Str_len|123456789.123456789.123456789.123456789}: {{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|123456789.123456789.123456789.123456789}}
}}}} }}}} }}}} }}}} }}}}
- Nest 29 templates & {Str_len|123456789.123456789.123456789.123456789}: {{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|{{Expansion depth limit exceeded|123456789.123456789.123456789.123456789}}
}}}} }}}} }}}} }}}} }}
Outside of the deep nesting, then the template runs fine:
- {{Str_len|123456789.123456789.123456789.123456789.1234567}}: 47
- {{Str_len|123456789.abcdefghi.123456789.123456789.123456789.12}}: 52
Templates designed for unnested logic
Fortunately, many common templates can be rewritten to unnest the if-else logic or avoid using too many embedded templates. Some examples of highly efficient templates are:
- {{str_left|str|n}} - extract left-side of a string for 'n' length
- {{strlen_short|str}} - count length of a string of 1-50 characters
- {{strfind_short|str1|str2|n|lenstr=m}} - search a string for another of 'n' length
- {{strloc_insert|str1|str2|strloc=n}} - insert string at 'n' or append when strloc <= 0
- {{strloc_prefix|string|strloc=n}} - extract prefix up to 'n' or whole string if strloc <= 0
- {{getprecision|x}} - determine the rounding precision of a decimal number
Those templates were specifically designed to use the minimal nesting of if-else logic and avoid too many embedded templates.
Relatively few understand the problems
Because not everyone working in Wikipedia has been trained in the limits of the MediaWiki software, even some of the admins might not be aware how the nested if-else logic has been limiting the use of templates inside of other large templates. Consequently, many people have tried to write templates as if they were writing computer software for modern computer systems, totally unaware of the unusual restriction of 40 levels of nested logic, where other computer software would allow 300, or perhaps unlimited, levels of nesting.