Significance Testing
The Significance card runs up to two independent statistical tests over the chart and decorates the points whose difference from a reference is statistically significant. The maths is shared with the SDM Smart Table visual, so the two product lines always agree on the same data.
Prerequisites
Significance testing needs a sample size. Bind a measure to the Base (sample size) data role — without it the Significance and Base Row cards do not appear in the Format pane at all.
| Data Role | Type | Role in the test |
|---|---|---|
| Base (sample size) | Measure | n for every test. Read per category, shared by all series at a given X. Required. |
| Standard Deviation | Measure | Per series and per point. When present, switches the test from proportions to means. Optional. |
Because the base is per category, all series compared at the same X use the same n.
Which test is run
The choice is automatic and made per comparison, not globally:
- Mean test — when both compared points carry a standard deviation.
- Proportion test — in every other case.
The Match status line inside each test tells you which one applies, so you can confirm your setup at a glance.
If your measure carries a percentage format string, its values are scaled from 0–100 back to 0–1
before the proportion test, so 45.3 % is tested as 0.453.
Card layout
The card is organised into three groups:
| Group | Contains |
|---|---|
| Base | Significance Level, Variance Method, and the two mode dropdowns Significance 1 / Significance 2 |
| Test 1 | Display options and the reference settings for the first test |
| Test 2 | Display options and the reference settings for the second test |
The two tests are fully independent — different comparison modes, different display modes, different colors — and both can decorate the same point at the same time.
Base group
| Property | Description | Default |
|---|---|---|
| Significance Level | 90 %, 95 %, 99 % | 95 % |
| Variance Method | Pooled Proportion, Separate — proportion test only | Pooled Proportion |
| Significance 1 | Comparison mode of test 1 | None |
| Significance 2 | Comparison mode of test 2 | None |
The confidence level maps to a two-sided critical value from the normal approximation: 1.645 at 90 %, 1.96 at 95 %, 2.58 at 99 %. A difference is reported as significant when the absolute test statistic exceeds that value.
The Variance Method only affects the proportion test — it chooses between a pooled standard error and a separate one. It is ignored when the mean test runs.
Comparison modes
Set Significance N in the Base group; the matching reference field then appears in the Test N group.
| Mode | Reference | Extra field shown |
|---|---|---|
| None | — test disabled | — |
| vs. Preceding category | The previous point of the same series | — |
| Regular expression on X | A category, per series | Regex on X (category) |
| Regular expression on Y | A whole series | Regex on Y (series) |
vs. Preceding category
Each point is compared to the point immediately before it, inside its own series. The first category is never decorated — it has no predecessor.
Regular expression on X
The pattern designates a reference category. Inside each series independently, the reference is the first point whose category name matches; every point of that series is then compared to it. All series therefore use a reference at the same position on the X axis, but each with its own value.
The reference point compares to itself, so it never carries a decoration.
Typical use: a baseline wave, a control cell, a "Total" category.
Regular expression on Y
The pattern designates a reference series. The reference is the first series whose name matches; at every X, all series are compared to that series at the same X. This mode needs a Legend field with at least two series.
Both sides of the comparison use the base of the current category, since the base is shared per X.
Series are paired by position in the category list, not by category name. This is the normal case in Power BI — all series share the same axis — but be aware of it if you build the visual from a source where series do not share the same categories.
Writing the pattern
The field takes a JavaScript regular expression, evaluated case-insensitively and as a partial match — the pattern only has to appear somewhere in the name, it does not have to match the whole string.
| Pattern | Matches |
|---|---|
2009 | 2009, Q1 2009, 2009-T4 … |
^2009$ | exactly 2009 |
^Total | any name starting with Total |
Ref|Base | any name containing Ref or Base |
Cat\. A | Cat. A — the dot is escaped to mean a literal dot |
Remember that ., +, *, ?, (, ), [, ], {, }, ^, $, | and \ are regular
expression operators and must be escaped with a backslash to be matched literally.
Match status
Directly under the pattern field, a read-only Match status line validates your input live:
| Message | Meaning |
|---|---|
(enter a regex) | The field is still empty |
✓ "Q1 2009" (mean test) | The pattern matched — the quoted name is the reference that will be used, and the test that will run |
✗ no match (proportion test) | Valid pattern, but nothing matched — the test stays inactive |
⚠ invalid regex | The pattern does not compile — the test stays inactive |
Display options
Each test group carries its own presentation settings.
| Property | Description | Default |
|---|---|---|
| Display mode | Icon, Border color, Background color, Marker | Icon |
| Positive color | Used when the value is significantly higher than the reference | #2E7D32 |
| Negative color | Used when the value is significantly lower than the reference | #C62828 |
| Custom icon positive (base64) | Replaces the default up arrow — Icon mode only | (empty) |
| Custom icon negative (base64) | Replaces the default down arrow — Icon mode only | (empty) |
Points that are not significant get no decoration at all.
Display modes
- Icon — a 12 × 12 arrow next to the point. When the series shows data labels, the icon is placed immediately to the right of the label; otherwise just above the point. When both tests are active and both use icons, they are laid out side by side so they never overlap.
- Border color — a rounded outline around the data label, or a small box around the point when the series has no label.
- Background color — a translucent filled rectangle behind the data label. It is drawn behind the text and takes precedence over the data label's own background, so the value stays readable.
- Marker — the data point itself is repainted in the significance color, with a white outline.
As soon as one active test uses Marker, the standard per-series markers are switched off for the whole chart, not just for the significant points. Use it when significance is the main message of the chart; prefer Icon or Border color when you also need your regular marker shapes.
Custom icons
The two icon fields accept either a full data URI (data:image/png;base64,…) or a bare base64
payload, which is then assumed to be a PNG. Anything else is rejected and the default arrow is used
instead. Icons are drawn at 12 × 12 pixels with their aspect ratio preserved.
When no custom icon is supplied, the default triangular arrow is generated on the fly and tinted with the Positive color / Negative color of that test — so in Icon mode those two colors still control the appearance.
Base Row
The Base Row card displays the sample size under the X-axis labels. It appears in the Format pane only when a Base field is bound, and the bottom margin of the chart is extended to host it.
| Property | Description | Default |
|---|---|---|
| Show | Master toggle | Off |
| Label | Text placed at the start of the row, left of the first value | Base |
| Font | Font family | Segoe UI |
| Text Size | Font size in pixels | 9 |
| Bold / Italic / Underline | Text style toggles | Italic on |
| Color | Text color | #888888 |
Each value is centred under its category. Leaving the Label empty removes it and keeps only the numbers.
When no result appears
A test silently produces nothing — rather than an error — in all of these situations:
- the Base value is missing, zero or negative at one of the two points;
- either value is blank;
- the standard error works out to exactly zero;
- the regular expression matched nothing, or did not compile;
- in vs. Preceding category mode, on the first category of the axis;
- in Regular expression on X mode, on the reference point itself.
Use the Match status line as your first diagnostic: it tells you whether the reference was resolved and which test is about to run.
Statistical conventions
For readers who need to reconcile the output with their own calculations.
Proportion test — the statistic is z = (p₁ − p₂) / se, with
- Pooled Proportion:
p = (n₁p₁ + n₂p₂) / (n₁ + n₂)andse = √( p(1−p) · (1/n₁ + 1/n₂) ) - Separate:
se = √( p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂ )
Mean test — the statistic is t = (m₁ − m₂) / se, with se = √( s₁/n₁ + s₂/n₂ ) where s is
the value supplied in the Standard Deviation role.
The mean test feeds the standard deviation directly into the variance slots of the standard error. This convention is inherited verbatim from the SDM Smart Table visual so that a line chart and a table built on the same measures always flag the same cells. If your reference implementation squares the standard deviation first, expect a difference on the mean test.
In both cases the difference is reported as significant when |statistic| is strictly greater than
the critical value of the chosen confidence level, and the sign of the difference selects the
positive or negative decoration.