{{ column.name or '(unnamed)' }}

01 / Summary

What this column holds

The figures of the column, and how Tabalyst reads its type.

Analyzable values: {{ column.field.values.count | count }}in {{ ds.summary.row_count | count }} rows
Missing: {{ column.missing_percent }}%{{ column.missing_count | count }} values{% set parts = sm.findings.missing.components.model_dump().items() | selectattr(1) | list %}{% if parts %}
{% for name, n in parts %}{{ name }} {{ n | count }}{% if not loop.last %} · {% endif %}{% endfor %}{% endif %}
Distinct: {% if d.kind == 'exact' %}{{ d.value | count }}{% elif d.kind == 'at_least' %}≥ {{ d.value | count }}{% else %}limited{% endif %}{% if d.kind == 'exact' %}{{ acc('exact') }}{% elif d.kind == 'at_least' %}{{ acc('at_least') }}{% if d.estimate is not none %} {{ acc('estimated') }} ≈ {{ d.estimate | round | int | count }}{% endif %}{% else %}{{ acc('not_computed') }}{% endif %}
Findings: {{ n_err + n_iss }}{{ n_err }} {{ 'error' if n_err == 1 else 'errors' }} · {{ n_iss }} {{ 'issue' if n_iss == 1 else 'issues' }}{% if n_rem %} · {{ n_rem }} {{ 'remark' if n_rem == 1 else 'remarks' }}{% endif %}

Interpretation

Type: {{ column.type_label }} {{ column.classification.state }} · {{ column.classification.basis }}
Technical type: {{ column.inferred_type }}{% if column.type_confidence is not none %} {{ ((column.type_confidence * 100) | round(1)) ~ '% confidence' }}{% endif %}
Traits: {{ column.classification.traits | join(', ') | replace('_',' ') }}
Alternatives: {% for alt in column.classification.alternatives %}{{ type_labels.get(alt.family ~ '/' ~ alt.type, alt.type | replace('_',' ')) }} {{ alt.reason | replace('_',' ') }}{% endfor %}
Evidence: {% for e in column.classification.evidence %}{{ e.id }} {{ e.role }}{% endfor %}
Exposure: {{ {'show': 'Shown', 'mask': 'Masked', 'hide': 'Hidden'}.get(column.exposure, 'Normal') }}

Type counts

{% for name, count in column.type_counts.items() %}

{{ name | replace('_',' ') }}{{ count | count }}

{% else %}

No classified values.

{% endfor %}

As stored in the source

{% for name, count in column.field.native_types.items() %}

{{ name }}{{ count | count }}

{% endfor %}

02 / Findings

Findings({{ n_err + n_iss + n_rem }})

What Tabalyst found in this column, each with what it counts, how sure it is and the evidence behind it. Errors first, then issues, then remarks.

{% if not (n_err + n_iss + n_rem) %}

{{ acc('exact') }} Nothing was found in this column.

{% endif %} {% for f in page.findings %}{% set p = f.preview %}

{% if p.share or p.population %}

{% endif %}

What it is: {{ p.nature }}
Counted in: {{ p.unit }}
Accuracy: {{ p.accuracy }}: {{ p.accuracy_note }}
Limit: {{ limit }}
Examples: {% for value in p.examples %}{{ value }}{% endfor %}{% if p.examples_note %}{{ p.examples_note }}{% endif %}
Examples: {{ p.examples_note or no_example }}

{% if f.detail or f.variants %}

{% if f.detail %}{{ detail_body(f.detail) }}{% else %}{{ variant_body(f.variants) }}{% endif %}

{% endif %}

{% endfor %} {% if page.remarks %}

Remarks not defects

{{ status(item.status_word, item.status_class, item.status_title) }} {{ item.title }}{{ acc(item.kind) }} · {{ item.noun }}{% if o.of is not none %} out of {{ o.of | count }}{% endif %}{% if item.kind == 'at_least' %}≥ {% endif %}{{ o.count | count }}{% if r.detail %}
{{ r.detail.title }}
{{ detail_body(r.detail) }}
{% endif %}

{% endif %} {% set checked = page.checks %} {% if checked %}

Checked, nothing found({{ checked | length }})

{{ line.name }}{{ line.state_label }}{% if line.detail %}{{ line.detail }}{% endif %}

{% endif %}

03 / Values

Values({{ ev.values | length }})

{% set shown = ev.values %}{% if shown %}{% set top = shown | map(attribute=1) | max %}

{{ value }}{% if value in ev.cut %}…{% endif %}{{ n | count }}

{% else %}

No stored values{% if column.exposure == 'hide' %}: the exposure setting of the column hides them{% endif %}.

{% endif %} {% if sm.details.measures %}

Measures of the Details column({{ sm.details.measures | length }})

{% for m in sm.details.measures %}

{{ m.label }}{{ m.value | count if m.value is number and m.value is integer else m.value }}

{% endfor %}

{% endif %}

{% if page.shape %}

04 / {{ page.shape.nav }}

{% if column.numeric %}

Minimum: {{ column.numeric.minimum | number }}
Maximum: {{ column.numeric.maximum | number }}
Mean: {{ column.numeric.mean | number }}
Median: {{ column.numeric.median | number if column.numeric.median is not none else 'limited' }}Range {{ column.numeric.range | number }}

{% endif %} {% if column.date_profile %}{% set date = column.date_profile %}

Valid: {{ date.valid_count | count }}{{ date.valid_percent }}%
Ambiguous: {{ date.ambiguous_count | count }}{{ date.ambiguous_percent }}%
Invalid dates: {{ date.invalid_date_count | count }}{{ date.invalid_date_percent }}%
Other values: {{ date.not_date_count | count }}{{ date.not_date_percent }}%

{% endif %} {% if column.string_profile %}{% set string = column.string_profile %}

Shortest: {{ string.minimum_length }}characters
Longest: {{ string.maximum_length }}characters
Mean length: {{ string.mean_length | number }}
Median length: {{ string.median_length | number }}{% if string.fixed_length is not none %}fixed length {{ string.fixed_length }}{% else %}{{ string.status | replace('_',' ') }}{% endif %}

{% endif %} {% for card in column.insights %}{{ insight_card(card, 'c-', 'h3') }}{% endfor %} {% if column.string_profile %}{{ character_map(column.characters) }}{% endif %}

{% endif %}

05 / Formats and patterns

Formats and patterns

The shapes the values take, once each: the formats the detectors recognized, then the patterns inferred from the values.

Formats

{% if page.formats %}{% for r in page.formats %}{% endfor %}

Format	Read as	Values	Share
`{{ r.format }}`	{{ r.read }}	{{ r.count \| count }}	{{ r.percent }}%

{% else %}

No format was recognized.

{% endif %}

{% if column.detectors %}

Detectors({{ column.detectors | length }})

{% for detector in column.detectors %}

{{ detector.id | replace('_',' ') | title }}{% if detector.primary %} primary{% endif %}

Status: {{ detector.status }}
Eligible: {{ detector.eligible_count | count }}
Matched: {{ detector.matched_count | count }} ({{ detector.matched_percent }}%)
Ambiguous: {{ detector.ambiguous_count | count }}
Invalid: {{ detector.invalid_count | count }}

{% if detector.evidence %}{% for state in ['matched','ambiguous','invalid','not_matched'] %}{% set values = exposed_evidence(column, detector.evidence | attr(state)) %}{% if values %}

Examples, {{ state | replace('_',' ') }}

{% for v in values %}{{ v }}{% endfor %}{% endif %}{% endfor %}{% endif %}

{% endfor %}

{% endif %} {% set pv = column.patterns %}{% if pv.status == 'complete' %}{% set e = pv.entry %}{% set kind = 'exact' if e.population.complete else 'sample' %}

Patterns({{ pv.rows | length }} in {{ pv.families | length }} {{ 'family' if pv.families | length == 1 else 'families' }}, {{ acc(kind) }} on {{ e.population.distinct | count }} distinct values)

Patterns are inferred from the distinct values the scan sampled: 9 a digit, A a capital, a a lowercase letter, @ any letter, X a capital or a digit, x a lowercase letter or a digit, # a letter or a digit. Quoted text is a constant.

Reading notes

Unmatched values{{ pv.outside_distinct | count }} · {{ (pv.outside_share * 100) | round(2) }}%

{% set x = e.population.excluded %}{% if x.long_text_occurrences %}

Long texts, not read{{ x.long_text_occurrences | count }}

{% endif %}{% if x.too_long_values %}

Values too long{{ x.too_long_values | count }}

{% endif %}{% if x.too_complex_values %}

Values too complex{{ x.too_complex_values | count }}

{% endif %}{% if x.empty_values %}

Empty values{{ x.empty_values | count }}

{% endif %}{% if x.non_string_values %}

Not text{{ x.non_string_values | count }}

{% endif %}{% if e.limits %}

Bounded by {{ e.limits | join(', ') }}.

{% endif %}{% if not pv.has_validation %}

Not validated against the whole file.

{% endif %}

{% elif pv.status != 'missing' %}

{{ acc('not_computed') }} {{ pv.reason_text }}.

{% endif %}

06 / Cleaning

Cleaning

How the values are normalized before they are compared. The raw values stay in the data sample.

{% for stage in column.normalization.stages %}{% endfor %}

Stage	Values changed	Distinct after
{{ stage.stage \| replace('_',' ') }}{% if not stage.enabled %} disabled{% endif %}	{% if stage.changed_count is none %}–{% else %}{{ stage.changed_count \| count }}{% if stage.changed_percent %} {{ stage.changed_percent }}%{% endif %}{% endif %}	{{ stage.distinct_count \| count if stage.distinct_count is not none else stage.distinct_status }}

Spelling variants ({{ column.normalization.variant_group_count | count if column.normalization.variant_group_count is not none else column.normalization.variant_group_status }} groups) are listed with the findings.{% if sm.finding_details.variants %} See Findings.{% endif %}

07 / Limits

Limits and accuracy

What the scan could not establish for this column, and how to read the accuracy marks.

{% if column.limits %}

{{ item.label }}{{ item.measure }} · {{ item.reason }}{% if item.setting %} · {{ item.setting }}{% endif %}{% if item.limit is not none %} · limit {{ item.limit | count }}{% endif %}{% if item.note %} · {{ item.note }}{% endif %}{% if item.lower_bound is not none %}{{ acc('at_least') }} {{ item.lower_bound | count }}{% else %}{{ acc('not_computed') }}{% endif %}

{% else %}

{{ acc('exact') }} No measure of this column was limited.

{% endif %} {% if diagnostics %}

Diagnostics

{{ diagnostic.message }}{{ diagnostic.code }} · {{ diagnostic.level }}{{ diagnostic.count | count }}

{% endif %}

{{ acc('exact') }}: {{ acc_titles['exact'] }}
{{ acc('at_least') }}: {{ acc_titles['at_least'] }}
{{ acc('sample') }}: {{ acc_titles['sample'] }}
{{ acc('estimated') }}: {{ acc_titles['estimated'] }}
{{ acc('not_computed') }}: {{ acc_titles['not_computed'] }}

{% set field = column.field %}{% set lengths = field.string_lengths.value if field.string_lengths.status == 'complete' else none %}{% set stats = field.numeric.value if field.numeric.status == 'complete' else none %}{% set booleans = field.booleans.value if field.booleans.status == 'complete' else none %}{% set temporal = field.temporal.value if field.temporal.status == 'complete' else none %}

08 / Scan evidence

Scan evidence

The facts of the field as the Scan wrote them. Open a group to read it.

Presence and missing

First record: {{ field.first_record if field.first_record is not none else '–' }}
Occurrences: {{ field.occurrences | count }}
Present: {{ field.presence.present | count }}
Absent: {{ field.presence.absent | count if field.presence.absent is not none else '–' }}

{% for name, value in field.missing.components.model_dump().items() %}

{{ name }}{{ value | count if value is not none else '–' }}

{% endfor %}

Missing definition: {{ field.missing.definition | join(', ') }}

First and last values

{% if field.values.first %}

First · record {{ field.values.first.record }}{{ field.values.first.value }} · {{ field.values.first.type }}

{% endif %}{% if field.values.last %}

Last · record {{ field.values.last.record }}{{ field.values.last.value }} · {{ field.values.last.type }}

{% endif %}{% if not field.values.first and not field.values.last %}

No visible values.

{% endif %}

String categories and characteristics

{% for name, value in field.strings.model_dump().items() %}

{{ name | replace('_',' ') }}{{ value | count }}

{% endfor %}{% set chars = field.string_characteristics %}{% for name, value in chars.model_dump().items() if value is number %}

{{ name | replace('_',' ') }}{{ value | count }}

{% endfor %}{% if lengths %}

Scan length histogram

{% for item in lengths.length_histogram %}

{{ item.length }} characters{{ item.count | count }}

{% endfor %}{% endif %}

{% if stats %}

Numeric statistics

Numeric values: {{ stats.count | count }}
Sum: {{ stats.sum | number }}
Variance: {{ stats.population_variance | number }}
Standard deviation: {{ stats.population_std | number }}
Positive / negative / zero: {{ stats.positive | count }} / {{ stats.negative | count }} / {{ stats.zero | count }}

{% endif %} {% if booleans %}

Boolean values

True{{ booleans.true | count }}

False{{ booleans.false | count }}

{% endif %} {% if temporal %}

Temporal range

{{ temporal.count | count }} recognized; {{ temporal.ambiguous | count }} ambiguous.

{% for kind in temporal.kinds %}

{{ kind.kind | replace('_',' ') }}{{ kind.min }} → {{ kind.max }}{{ kind.count | count }}

{% endfor %}

{% endif %}

{% if ds.rows.rows %}

09 / Data sample

Data sample

{% set rc = ds.rows.counts %}

{{ rc.start | count }} rows at the start, {{ rc.middle | count }} sampled from the interior and {{ rc.end | count }} at the end, with their original row numbers. The sample is not uniform over the file.

Row {{ row.record }}{% for origin in row.origins %}{{ origin_words[origin] }}{% if not loop.last %} · {% endif %}{% endfor %}{% if column.id not in row.values %}absent{% elif row.values[column.id] is none %}hidden{% else %}{% set cell = row.values[column.id] | cell_text %}{% if cell == '' %}empty{% else %}{{ cell }}{% endif %}{% endif %}

{% endif %}

{{ section.title }}

What this column holds

Interpretation

Type counts

As stored in the source

Findings({{ n_err + n_iss + n_rem }})

{{ p.title }}

{{ f.detail.title if f.detail else 'Spelling variants' }}

Remarks not defects

{{ r.detail.title }}

Values({{ ev.values | length }})

{{ page.shape.title }}

Formats and patterns

Formats

{{ detector.id | replace('_',' ') | title }}{% if detector.primary %} primary{% endif %}

Reading notes

Cleaning

Limits and accuracy

Diagnostics

Scan evidence

Scan length histogram

Data sample