How to automate POS data analysis

Scheduling the retailer exports is the smaller half of the job. The product mapping and the calendar underneath them are where the week goes.

Kartik Deshpande··7 min read

The weekly routine starts in a browser: log into each retailer portal, run the same export you ran last week, save it beside the distributor spreadsheet that arrived by email and the marketplace report, then spend the morning making the three agree. Automating POS data analysis means those feeds land on a schedule, every product code resolves to one product before anyone looks at it, and the weekly questions run against that instead of against a workbook rebuilt each Monday.

For one retailer and one distributor, a workbook holds. At four retailers, two distributors and a marketplace, the reconciliation grows with the number of feeds rather than with your familiarity with them, and the number a buyer quotes in a meeting stops matching the number in your own deck.

Where the weekly routine breaks

Every feed describes your product its own way. A retailer reports its internal item number against a store number, a distributor reports cases shipped out of a warehouse, and a marketplace reports units ordered against its own identifier. None of them uses your SKU, so the first hour goes on rebuilding the same mapping as last week, usually with a lookup against a sheet one person maintains by hand.

The calendars disagree next. Retailer files tend to run on a retail week ending Saturday, grouped into fiscal months of four and five weeks rather than calendar months. Marketplace settlements land well after the order. Your own shipments sit in NetSuite on a normal month. A weekly sales figure assembled across the three describes three different periods and presents as one.

What gets lost in the assembly is the difference between selling more and selling in more places. Units rising because a chain expanded the store list is a distribution gain; units rising at a flat store count is velocity. A workbook that tracks totals cannot separate them, so a promotion reads as a success in the week a distribution change happened to land.

Then the files move under you. A retailer adds a column, renames a measure, or changes which stores appear, and a workbook that depended on column order returns a number that is wrong rather than one that is missing. Wrong numbers survive review, which is why consumer brands usually discover the break in a buyer meeting rather than on the Monday it happened.

One product as three feeds describe it
Retailer POS
  • Item and store number
  • Units off the shelf
Weekly, in arrears
Distributor
  • Case pack, warehouse
  • Cases shipped in
Not sell-through
Marketplace
  • Its own identifier
  • Units ordered
Settles later
Each feed identifies the product differently and measures a different event, so the mapping and the calendar have to be settled before any two can be compared.

What automating POS data analysis requires

Scheduling a download does not make the analysis automatic. Four things have to be true before a weekly question can be answered without assembling it first.

The feeds have to arrive without a person. Some portals offer a scheduled export or a file drop; others only let you click a button. Where the click is unavoidable, the automation starts at the point the file lands, which means one location and a stable name rather than a download folder and a date in the filename.

Every code has to resolve to one product. Retailer item number, barcode, distributor case pack, marketplace identifier, and your own SKU all have to point at the same row, with the conversion from cases to eaches recorded against the pack rather than reapplied per file. This is the part that decides whether two feeds can be added together at all.

The calendar has to be declared. Choose the week you report on, then record what each feed lags by, so a four-week comparison that includes a feed running two weeks behind either waits for it or says which account is missing. A gap that announces itself costs less than a complete-looking period that is short one retailer.

The measures have to be defined once. Sell-through, shipments, velocity and weeks of supply each answer a different question, and each has more than one defensible definition. Write the one your team uses down somewhere the reports read it, not in the header of a tab.

MeasureWhat it answersWhere it comes from
Sell-throughWhat moved off the shelfRetailer POS feeds, or syndicated data
ShipmentsWhat left your warehouseYour ERP, plus distributor reports
Velocity per storeWhether demand grew or distribution didPOS units against stores selling
Weeks of supplyWhether an account is about to go out of stockRetailer inventory against recent sell-through

The steps to automate it

Step 1: List every feed you receive

Portal exports, distributor spreadsheets, marketplace reports, and syndicated data if you buy it. For each one, write down how it identifies a product, what period it covers, how late it arrives, and who is allowed to see it. Start with the two that disagree most often (the distributor report and your own shipments) before touching the rest, because that pair is where the reconciliation hours are.

Retailer files often carry contractual limits on distribution, so access control belongs on this list rather than in a later phase: a broker working one account should see that account and not the others. How the data is handled covers per-person and per-dataset scoping, including what an AI agent is allowed to read.

Step 2: Build the product key first

Nothing downstream survives until one product is one row. Automated matching gets the easy cases, where a barcode or a description lines up across two feeds. The remainder are commercial decisions rather than lookups: whether a discontinued variant rolls up into its replacement, how many eaches are in this year's case, whether a club pack counts as one unit or three. Each decision has to be recorded where it gets applied again next week instead of re-argued.

Step 3: Declare the calendar and the lag

Pick one week definition for reporting and convert everything into it, including your own shipments. Record each feed's lag beside it. The test of this step is whether last week's number changes when a late feed arrives: it should, and the report should say which account moved it, rather than replacing a figure someone has already sent to a buyer.

Step 4: Define the measures, then stop redefining them

Sell-through gross or net of returns. Velocity per store or per point of distribution. Whether a promotional week is comparable to a base week at all. These are decisions your commercial and finance teams may answer differently, and the argument is worth having once, in writing, rather than every time two decks disagree.

We built Permute for this shape of problem. We connect the systems the data lives in, including Shopify, Amazon Seller Central, NetSuite and QuickBooks alongside the retailer and distributor files that only ever arrive as spreadsheets, and resolve them into one record per product across every feed it appears in. The Ontology layer is where the definitions then live as explicit rules: the case pack conversion, the reporting week, and what sell-through means net of returns are written once and applied wherever they are used, so a question asked in plain language, such as which accounts grew last month on a flat store count, is answered against your stated definition rather than one inferred on the spot, and the answer names the rows it came from. Because those rules live next to the data rather than in a workbook, an AI assistant pointed at the workspace reasons from the same definitions your team agreed. We read from these systems and do not write back into them, so a barcode entered wrong is still corrected in your ERP (we surface which rows disagree, which is usually the slow part of finding it). Typical implementations go live in weeks rather than quarters.

See whether your feeds agree

Connect one retailer feed and your own shipments for a single product line, declare the case pack and the reporting week, and ask for last month. Any gap between the two numbers is the work.

Step 5: Let the exceptions stay human

Automation earns its place by raising the exceptions rather than smoothing them: a store list that shrank, a week that never arrived, one barcode appearing under two products, a promotion that ran on different dates than the calendar recorded. None of those has a correct default. What changes is that they arrive as flags on Monday instead of as a discrepancy someone notices in a quarterly review.

What the week looks like once it runs

The feeds are in before anyone opens a browser, the mapping already holds, and the questions are asked rather than assembled. A figure quoted in a buyer meeting reconciles to the rows behind it, so a challenge to it is a lookup instead of a rebuild. A question about one account three days later is a question.

Two limits are worth stating. We are not a forecasting engine, so the replenishment call still belongs in a planning tool, working from history that is now complete and defined. And syndicated data still answers what your own feeds cannot, which is what happened in the category around you. CPG analytics software: what to look for before you buy covers which category of tool does which part of this, and what a context layer actually does covers the mechanism underneath.

Test it on one account

Pick the retailer whose numbers you argue about most, connect it beside your shipments, and see where the two stop agreeing.

Questions consumer brands ask about POS data

What if a retailer portal only allows manual downloads?

Then the download stays manual and everything after it does not. Someone clicks the button, the file lands in one agreed location, and the mapping, the calendar conversion and the measures apply from there. The weekly hour that disappears is the assembly, not the click, and a portal that later offers a scheduled export changes only the first step.

How much POS history should we load?

Enough to cover a full year of seasonality plus the year before it, so a comparison has something to compare against. Older history is useful but costs mapping decisions, because it contains discontinued items, retired barcodes and pack sizes you no longer sell. Load it when a specific question needs it rather than on principle.

Does this replace syndicated data?

No. Your own feeds tell you what your products did; syndicated data tells you what the category did around them, including competitors you have no other view of. We can bring a syndicated extract in as one source alongside retailer and distributor feeds, and we do not sell that data ourselves.

Is it cheaper to build this in house?

It depends on whether the mapping is a project or a standing job. The connectors and the scheduled loads are a project an analyst can finish. The product key, the calendar rules and the measure definitions need maintaining every time a retailer changes a file or you launch a pack, which is why the in-house data function a mid-market company would need runs past $2M a year, and outside help benchmarks at $300 an hour and up. What it costs is public, and the comparison worth making is against the maintenance rather than the build.