Context: Reporting work often starts from messy
exports. Before those files feed a dashboard or a SQL load, they
need a pass/fail check for schema, nulls, and duplicate rows.
What I did: Wrote a single Pandas script that loads
a sample orders CSV as text, then runs a contract of rules: expected
columns and types, required fields, full-row duplicates, duplicate
order IDs, quantity greater than zero, and allowed status/product
values. Failed checks print Excel-style row numbers so the source
file can be opened and fixed.
Outcome: Against a 10-row sample with planted
issues, the checker passed column names and failed the seven
remaining rules (types, nulls, duplicates, range, and allowed
values). A non-zero exit code marks the run as failed for a
scheduled job.
[PASS] Schema - column names
[FAIL] Schema - value types
[FAIL] Nulls - required fields
[FAIL] Duplicates - full rows
[FAIL] Duplicates - business key
[FAIL] Range - quantity > 0
[FAIL] Allowed values - status
[FAIL] Allowed values - product
SUMMARY: 1 passed, 7 failed
Terminal report from the sample file: planted issues in dates, prices,
blank customer IDs, duplicate order 1003, negative quantity, and typos
like SHIPPD.
Exploratory analysis and Excel automation with Pandas and NumPy.
The
data quality checker
is written up as its own case study above. Highlighted notebooks
below; more scripts live in the
Python folder.
Production-style reporting in SQL Server Management Studio (SSMS):
multi-table joins, CASE logic, and filters for HR / ops exports.
Sample queries are in the
SQL folder.
Ops org chart query
— employee master + position + supervisor joins for a reporting hierarchy.
SELECT DISTINCT
RIGHT(PRPMS.PREN, 6) AS EmployeeID,
PRPMS.PRCKNM AS Name,
PRPMS.PRL03 AS Dept,
PRPMS.PRTITL AS Title,
PRPMS_1.PRCKNM AS Reports_to
FROM psa.dbo.SMFD35_HRDBFA_PRPMS PRPMS (NOLOCK)
LEFT JOIN psa.dbo.SMFD35_HRDBFA_PEPOG PEPOG
ON PRPMS.PRER = PEPOG.OGER AND PRPMS.PRPOS = PEPOG.OGPOS
LEFT JOIN psa.dbo.SMFD35_HRDBFA_PRPSP PRPSP (NOLOCK)
ON PRPMS.PRER = PRPSP.SPER AND PRPMS.PREN = PRPSP.SPEN
LEFT JOIN psa.dbo.SMFD35_HRDBFA_PRPMS PRPMS_1 (NOLOCK)
ON PRPSP.SPSPER = PRPMS_1.PRER AND PRPSP.SPSPEN = PRPMS_1.PREN
WHERE PRPMS.PRL01 = '001'
AND PRPMS.PRL02 = 'XTT'
AND PRPMS.PRTEDH = '0'
AND PRPMS.PRSEC <> 'DRV'
ORDER BY PRPMS.PRCKNM ASC;
Excerpt from the ops org chart query: supervisor self-join pattern used for hierarchy reports.
Power BI
Desktop dashboard screenshot (interactive embed needs Power BI service publish-to-web).