Donaldson R F
Data Conversion & Parsing
CSV Data Cleaner Script
Clean up a messy CSV automatically: duplicates removed, text formatted consistently, blanks handled sensibly.
The Problem
Spreadsheets built up over time accumulate duplicate rows, inconsistent capitalisation, and trailing whitespace that quietly breaks lookups and reports — cleaning it manually is tedious and error-prone.
About this problem
Spreadsheets that grow over time collect near-duplicate rows, inconsistent capitalisation, stray spaces and blanks. These quietly break lookups, pivot tables and imports because two values that look the same to a person are different to a computer.
People usually come to this before importing data into another system or running a report, when they find that totals or matches are wrong and the cause is hidden formatting rather than genuinely different data.
What's Included
- A script that removes duplicate rows based on the criteria you specify
- Standardising text formatting (case, whitespace, common typos if patterned)
- Sensible handling of blank/missing values
- A before/after summary of what was changed
What's NOT Included
- Deciding which of two conflicting "duplicate" records is correct — flagged to you rather than guessed
- Validating data accuracy against an external source (separate request)
- Ongoing automatic cleaning unless set up as a standing tool
How It Works
- I inspect your CSV to identify duplicate patterns, formatting problems and blank values.
- I agree with you which columns define a duplicate and how blanks should be treated.
- I write a script that trims whitespace and standardises case and other patterned formatting.
- I remove exact or rule-based duplicates and write ambiguous cases to a separate file for you to review.
- I produce a cleaned CSV, leaving the original untouched.
- I include a before and after summary of what changed.
In practice: you buy the service, send over whatever access or details the job needs, I investigate and do the work, and you confirm it's resolved before we call it done.
Frequently Asked Questions
- How do I remove duplicates and clean a CSV file?
- A script can trim spaces, standardise text and remove duplicates by the rules you choose. I build it around your data.
- What if duplicate rows disagree with each other?
- They are flagged for your decision. I do not guess which is correct.
- Will you change my original file?
- No. A new cleaned file is produced.
- Can it correct spelling mistakes?
- Only patterned ones that follow a clear rule. General spellchecking is not included.
- Does it verify the data is accurate?
- No. Validating against an external source is a separate request.
- Can it clean new files each week?
- You can run it again, but automatic cleaning is a standing tool and not included.