Choosing and running the software · 4.4

Bringing historical data across

Bringing historical data across. What decides it, what it costs, and what usually goes wrong. For another implementation reference, see Monitask's reference.

How much history do you actually needThe question to settle first

Less than the instinct suggests. The questions historical time data answers are mostly about trends and about supporting claims that are still live.

Establish what you need it for before deciding how much to move: an open invoice dispute, a statutory retention obligation, a capacity trend. Each implies a different amount and a different fidelity. For related guidance and industry context, see 8x8.

Four options, in increasing cost

Export to files and archive them. Cheapest, and adequate where the obligation is to retain rather than to query.

Keep the old system read-only for the retention period. Costs a licence and no effort.

Migrate summaries. Monthly totals by category and person, which supports trend analysis at a fraction of the work.

Migrate every entry. Expensive, and the option most organisations start by assuming they need.

Why full migration disappoints

The categories differ. Old entries have to be mapped onto new categories, the mapping is approximate, and the resulting history compares two things that were never the same.

Which means the analysis the migration was performed for is distorted anyway, and it would have been more honest to keep the two eras separate and say so.

Cut over at a period boundary

The start of a month, a quarter or a financial year. Mid-period cut-overs produce a payroll run and an invoicing cycle drawn from two systems, which is where the errors are.

Where a parallel run is possible, run one period in both. It is double entry for a month and it is the cheapest possible verification.

Verify by reconciling totals

Not by spot-checking records. Compare hours per person per month across the two systems for the migrated period, and investigate any difference rather than accepting it as rounding.

Migrations that were never reconciled are discovered years later, usually by an auditor asking why a figure does not match.

What to tell people

That history is moving, what is coming across, what is not, and where the old records will live. Somebody will need an entry from two years ago and should know where to look.

Keeping the old system

Longer than you expect. Decommissioning on the day of cut-over removes the fallback at exactly the moment it is most likely to be needed. Three months read-only is a small licence cost against a real risk.

Two eras, stated honestly

Where the categories changed, say so in the reporting: this figure covers the period before the change, on the old definitions. It is less tidy than a continuous line and it is the only honest presentation.

A continuous chart across a definitional break is a chart of two different things, and somebody will draw a conclusion from the step in it.

Who does the work

Migration is generally a supplier service, an internal project or a consultant. Whichever it is, somebody internal has to define the mapping, because only they know what the old categories meant.

That definition is the migration. The data movement is mechanical.

Timing against the year end

Avoid migrating in the weeks around a financial year end or an audit. Both need clean data from a stable system, and both are conducted by people who will remember a migration that made their month harder.

What to do with what you do not migrate

Export it, verify the export opens, store it where somebody will find it, and record in writing what it contains and until when it must be kept. Twenty minutes, and it is the difference between an archive and a forgotten file.

A realistic plan

Decide what history is needed and why. Map the categories, in writing, with somebody who knows what the old ones meant. Migrate a sample and reconcile it. Migrate the rest. Reconcile totals by person and month. Keep the old system read-only for a quarter. Record what was and was not moved.

Seven steps, and the two reconciliations are the ones that get dropped under time pressure and are the reason migrations are discovered to have been wrong years later.

Corrections and approvals

Historical entries frequently carry approval status and a correction history. Both are part of what makes a record evidence, and both are commonly lost in migration because the target system models them differently.

Ask specifically whether they come across. If they do not, that may be a reason to keep the old system rather than to accept the loss.

The people question

Leavers. A migration keyed on current employees silently drops everybody who has left, which is precisely the population a payroll audit is most likely to ask about.

Check the count of distinct people before and after. It is one query and it catches this immediately.

Where to start

Answer the first question honestly: what do you need history for. Most organisations find the answer is a statutory obligation and one trend, which together require far less migration than the instinct to move everything.

One line to carry

Move less than you think, map the categories in writing, and reconcile totals rather than spot-checking rows.

Sampling before committing

Migrate one month, one team, and look at it properly. Every mapping problem in the full migration is present in that sample, and finding it there costs an afternoon rather than a reversal.

One line to carry, again

The mapping is the migration. Everything else is moving bytes, and only somebody who knows what the old categories meant can write it.

Also in choosing and running the software