Data collection and management

Why it matters

Poor data costs money in ways that rarely appear on an invoice. Appointment reminders that fail because a quarter of mobile numbers are missing. Marketing sent twice to the same person under two spellings. Stock reordered because the count was wrong. Hours spent each month reconciling two lists that should have been one. Good data management is also a compliance matter: if you hold personal information you are expected to keep it accurate, secure and no longer than you need it, and a data breach can be a notifiable event.

Decide what to collect before you collect it

The instinct is to collect everything in case it is useful later. That instinct creates long forms customers abandon, staff who skip fields, and a store of personal information you are responsible for protecting but never use. A better test is to ask, for each field, what decision it will inform and how hard it is to collect well.

Fields that inform a real decision and are easy to collect go in now. Fields that are valuable but hard to collect, such as the source of every enquiry, deserve a plan: a dropdown, a training session, an automated capture. Fields that are easy but not useful should be left out unless the system already fills them. Fields that are hard and not useful should never be collected, especially if they are personal.

Capture it once, at the source

Most data errors are born at the moment of entry. The same customer typed in again because the search was slow. A job type written in free text in six different ways. A phone number with a missing digit nobody will notice until the reminder fails. The remedy is to make the right entry the easy one: dropdowns instead of text boxes, required fields for the things you actually need, format checks on emails and phone numbers, and a search-before-create step for customers and products.

Enter data once, in the system that owns it, and let other systems take it from there. Re-keying from one system to another is where duplicates and transcription errors multiply. Where systems need the same record, connect them with an integration rather than a person.

• Dropdowns and codes for anything you will group or filter on.
• Required fields only for data a decision depends on.
• Format checks on phone, email, ABN and postcode.
• Search before create for customers, suppliers and products.
• One owning system per kind of record; integrate rather than re-key.

Keep it clean with a routine

Data decays. Customers move, change numbers and close businesses; products are discontinued; staff leave and their open jobs sit unassigned. Without a routine, a list that was clean a year ago will be a quarter wrong today. A short monthly hygiene session, owned by one person, keeps it manageable: find duplicates, fill the gaps that matter, archive what is dead.

Measure the result. The share of records with complete key fields, the duplicate rate and the number of bounced emails or failed SMS messages are simple indicators that tell you whether the routine is working, and they make a useful addition to the operations dashboard.

Protect it and know when to let it go

Business data should be backed up automatically, stored in systems with access controls, and visible only to people who need it for their work. Personal information about customers and staff deserves extra care: limit who can export it, use strong authentication, and do not keep copies in spreadsheets on laptops.

In Australia the Privacy Act 1988 and the Australian Privacy Principles set out how organisations covered by the Act must collect, use, secure and dispose of personal information. Not every small business is covered, but some are regardless of size, including health service providers and businesses that trade in personal information, and the rules have been under reform. Check the Office of the Australian Information Commissioner's current guidance for your situation. Whether or not you are covered, collecting only what you need, telling people why, and deleting it when it is no longer required is good practice and reduces the damage if a breach ever happens.

How to get your data under control

1. Inventory your records List each kind of record, the system that owns it, who can access it and whether it contains personal information.
2. Decide what to collect Run each field through the value-and-effort grid. Remove what does not inform a decision.
3. Fix the capture Dropdowns, required fields, format checks and search-before-create in each owning system.
4. Stop re-keying Connect systems that need the same record so it is entered once.
5. Clean the backlog once Merge duplicates and fill gaps on active records. Archive the rest.
6. Set the monthly routine One owner, one checklist, one hour. Track completeness and duplicate rate.
7. Review retention and access Agree how long each kind of record is kept and who can see it. Check your Privacy Act position with the OAIC guidance.

Real-world example

An Adelaide physiotherapy clinic cleans up its records

Torrens Physio has four practitioners and about 3,800 patient records in its practice management system. Reception has always typed new patients in quickly, and the clinic has never run a clean-up. When the owner, Sam, tries to launch SMS appointment reminders, the problems surface: around 11 per cent of records are duplicates, and 22 per cent have no mobile number at all. As a health service provider the clinic is covered by the Privacy Act, so Sam is also uneasy about the old paper intake forms in a filing cabinet and the spreadsheet of patient emails on the reception laptop.

The clinic spends a month merging duplicates, makes mobile number a required field with a format check, adds a search-before-create step to the intake process, moves the email list into the practice system and shreds forms older than the retention period its professional body and privacy obligations require. Reminders go live to a clean list. No-shows fall from about 9 per cent to about 5 per cent of appointments. With roughly 120 appointments a week, that is about five recovered appointments weekly at $95 each, close to $24,000 a year, from data the clinic already had.

Common mistakes

• Collecting everything just in case Long forms get abandoned and skipped, and unused personal data is pure liability. Collect what a decision needs.
• Free text where a dropdown belongs Six spellings of the same job type make grouping impossible. Use codes and lists for anything you will filter on.
• Re-keying between systems Every manual copy is a chance for a duplicate or a typo. Integrate systems that share records.
• A one-off clean-up with no routine Data decays within months. A short monthly session keeps it clean; a yearly blitz does not.
• Personal data in spreadsheets on laptops Exported lists escape your access controls and backups. Keep personal information in the owning system.

KPIs to track

• Key-field completeness Share of active records with all required fields filled. Aim above 95 per cent.
• Duplicate rate Duplicates as a share of customer or product records. Under 2 per cent after clean-up.
• Bounce and failure rate Emails bounced and SMS messages failed as a share sent. Rising rates mean contact data is decaying.
• Records with personal data outside owning systems Count the exports and spreadsheets. The target is zero.

Frequently asked questions

Does the Privacy Act apply to my small business?

It depends on turnover and the kind of business. Some small businesses are covered regardless of size, including health service providers and those that trade in personal information, and the rules have been changing. Check the OAIC's current guidance or ask a professional adviser.

How long should I keep customer records?

Keep what you need for the purpose you collected it, plus any period required by law; tax records, for example, have statutory retention periods set by the ATO. After that, personal information should be deleted or de-identified. Agree a retention period for each kind of record and write it down.

What is the difference between a CRM and a spreadsheet of customers?

A CRM enforces structure (one record per customer, defined fields, history of interactions), supports multiple users safely and can connect to other systems. A spreadsheet does none of that reliably once more than one person edits it.

Who should own data quality?

One named person per kind of record, usually whoever uses it most. They run the monthly routine and are consulted before anyone adds a new field or system.

Key takeaways

• Decide what to collect by the decision it informs, not by what might be useful one day.
• Make the right entry the easy one: dropdowns, required fields, format checks, search before create.
• Enter each record once in its owning system and integrate rather than re-key.
• A short monthly hygiene routine with an owner keeps data clean; one-off blitzes do not.
• Protect personal information, keep it only as long as needed, and know your Privacy Act position.