A first look at a dataset

Every number you report comes from a call in this run. Do not read the raw file with a file tool, do not judge from the first rows, and do not fill a gap from what you know about a famous dataset: if you did not measure it, measure it or leave it out.

1. Measure

1. duckdb.inspect, with the data file's path as file: columns, row count, the first rows, and for a workbook its sheets.
2. profiler.profile with { "path": "<path>" } for a csv, tsv or parquet file. For a workbook or a json file, profiler.profileQuery with { "sql": "select * from <view>" }, using the view name duckdb.inspect returned.
3. Ask duckdb.query only for what the profile does not answer, with the file registered through {{file}}:
   • duplicate rows: select count(*) - count(distinct (t.*)) from {{file}} t;
   • the balance of a target the person named: select <target>, count(*) from {{file}} group by 1.

2. Read the profile

Go through every column, not only the interesting ones. Read nullPct, uniqueRatio, kind, min/max, q50 and topK. distinct is an estimate and can exceed the row count; judge uniqueness by uniqueRatio.

Flag a column when:

Finding	Rule
identifier	uniqueRatio is 1 (or above 0.99) and it names a row, not a quantity; it must not be a feature
mostly missing	nullPct above 50
some missing	nullPct above 0; give the exact percentage
constant	one value in topK, or min equals max
high-cardinality text	categorical with uniqueRatio above 0.5
skewed	numeric with max far above q75 (more than ten times the interquartile range)
suspicious zero or negative	a quantity whose min is 0 or below where that cannot be real

Do not call a target imbalanced unless the minority class is under 20%.

Two mistakes to avoid:

• Every column whose nullPct is above 0 is listed as missing, however small; never call a column complete without reading its nullPct.
• Say what a code means (1 is survived, 3 is third class) only when a column name or the person says so; otherwise give the values and their counts.

3. Report

Read report.md (call this skill with { "file": "report.md" }) and write the answer in that shape, in the person's language. Keep column names as they are in the file.

4. Offer a contract

End by offering a data contract that pins what you measured. When the person says yes, call mlops.pandera.contractFromProfile with the path and write: false first and show the draft; write it only after they confirm.