# F1-dev-01 — Firm cross-section: analytic dataset and revenue aggregates

# Question

How many distinct firms are in data/firms.csv (simulated data)? What are the mean and median annual revenue in millions of USD over firms with known revenue? What is the mean head count over all distinct firms? Every removed or transformed row is documented.

# Data and exclusions

• Input: data/firms.csv, 445 data rows, 5 columns. No other input was used.
• Problems found (see out/exclusions.csv and data_issues in results.json):
  1. Duplicate rows: 45 rows exactly repeat another row in every field (45 firm_ids appear twice). They were dropped, keeping one row per firm. No firm_id had conflicting values. 445 → 400 rows.
  2. Mixed units: 60 firms report revenue in USD_thousands; the other 340 use USD_millions. These were divided by 1000 and no rows were removed. After conversion, the two groups are on the same scale (raw medians are about 124.8 thousand-units and 178.0 million-units), which supports the per-row unit labels.
  3. Missing revenue: 6 firms (F0048, F0121, F0129, F0269, F0348, F0399) have empty revenue. They are excluded from the revenue aggregates but kept for the head-count mean, which the task defines over all distinct firms. 400 → 394 rows.
• Other checks found no problems: no unparseable numbers, no missing employees, no unknown units, all firm_ids have the same format, and there are 5 sector labels.

# Method

out/analysis.py reads the file as strings, trims whitespace and drops exact duplicates. It fails loudly if any firm_id still has conflicting rows. It converts revenue to millions of USD using each row's unit, computes the mean head count over all 400 firms, and drops firms with empty revenue. It then computes the mean and median revenue over the 394 remaining firms. As a baseline, it also takes the mean of the raw revenue column (non-empty rows, before any cleaning and with mixed units). Uncertainty comes from the standard error of the mean and a 5000-draw nonparametric bootstrap (seed 20261030). The script is deterministic: two runs gave byte-identical results.json.

# Results

The raw baseline is about 120 times the cleaned mean. This is almost entirely because thousand-unit values were mixed with million-unit values; the duplicate rows have a minor effect. The raw figure has no meaningful unit.

# Uncertainty

• Mean revenue: standard error 13.82 M USD, bootstrap 95% interval [224.84, 278.05] M USD.
• Median revenue: bootstrap 95% interval [144.74, 188.57] M USD.
• These intervals only matter if the firms are viewed as a sample from a larger population. As descriptions of this file, the aggregates are exact.
• Revenue is right-skewed (maximum 2057.6 M USD), which is why the mean is well above the median.

# Limitations and open issues

• The data are simulated, so nothing here describes real firms.
• Duplicates were treated as assembly errors because they are identical in every field. Two genuinely different firms with identical records could not be told apart.
• The 6 firms with missing revenue are excluded under a complete-case approach, with no imputation. If revenue is missing for reasons related to its size, the aggregates could be biased. With 6 of 400 firms missing, the effect is probably small but cannot be tested.
• Unit labels are assumed correct. The scale check supports this but cannot rule out mislabelling of individual rows.
• Process note: during development, one shell command ran from inside data/ and created a temporary data/out/analysis.py. It was moved to out/ and the empty folder removed straight away. data/firms.csv was never changed, and data/ again contains only that file.