Initpc E-commerce ERP: Syncing More Than 280,000 Products

An e-commerce store with more than 280,000 products in its catalogue does not come from a spreadsheet uploaded once and forgotten. It comes from price lists issued by different suppliers, from warehouses that update stock availability at their own pace, and from a shop that must always show a credible price and a credible stock level. That is the Initpc case: a catalogue of office supplies, stationery, IT equipment, school products and gift items, fed by several sources, with a single storefront on display.

The heart of the project is not WooCommerce, the current platform, but what sits upstream of it: a proprietary ERP / back-office system, developed by us, that ingests the price lists, normalizes them, decides what gets published and keeps prices and stock availability aligned across suppliers, warehouses and website. Without that layer, a catalogue of this size cannot be governed.

Here we explain how the pipeline works, what can go wrong at each stage and how we handle it. This is a deep dive from the Initpc case study, where you will find the overall architecture and the other chapters.

Why it cannot be done by hand, nor with an import plugin

The initial temptation is always the same: a CSV import plugin, a column mapping, a scheduled upload. It works with a single supplier and stops working as soon as several price lists come into play, because a plugin answers only one question — "how do I get these rows into WooCommerce?" — while the real questions are different. Who decides which supplier wins when the same product appears in more than one price list? What happens to the online price when the cheapest supplier runs out of stock? Where does the record of what was discarded end up?

Doing it by hand, across more than 280,000 SKUs, is not an option: every manual intervention is a decision that leaves no trace and cannot be repeated. What was needed was a system with its own logic, separate from the shop, that treated WooCommerce as a destination rather than as the place where the data gets tidied up.

The proprietary ERP: the brain upstream of the shop

The ERP / back-office system we developed for Initpc is a standalone application, with its own database and its own processes, sitting between the suppliers and the website. WooCommerce receives data from it that is already clean and already decided: it does not know the original price lists and makes no commercial decisions. The business rules — margins, supplier priorities, exclusions — live in one place and can be changed without touching the shop; the website can be replaced, as has already happened in Initpc's history, without rebuilding the supplier integration; errors stop upstream, where they can be examined, instead of reaching the storefront.

It is the same approach we follow when we design custom ERP and CRM software: the reference data lives in a system that is responsible for it, and everything else is a projection of it.

From price lists to the internal format

Ingestion: different formats, one pipeline

Each supplier delivers its price list however it prefers: tabular files with columns that change name from one version to the next, fixed-width layouts inherited from previous-generation ERPs, different character encodings, feeds to download, stock availability published separately. The ERP has a connector for each source, which fetches the file, interprets it, recognizes whether it is complete or truncated and always produces rows in a single internal format. From there on the pipeline is identical for everyone, and adding a supplier means writing a connector, not rebuilding the system.

Normalization: making comparable what is not

• Codes and identifiers. The supplier's code is kept as a reference, but the key that joins the price lists is the EAN, where one exists and is valid; an incorrect EAN, or one reused for different products, is flagged and not used as a key.
• Selling units and packs. One supplier sells the single piece, another the pack, another the carton: price and stock availability are brought back to the unit the shop actually sells, otherwise comparing sources makes no sense.
• Prices. The purchase price is read together with its conditions (volume discounts, promotions, ancillary charges) and from there the selling price is calculated according to the margin rules for each category.
• Attributes and categories. Colour, size, brand, compatibility: every supplier has its own vocabulary. Normalization translates it into the catalogue's vocabulary, and the shop's categories come from a mapping of our own, not from the supplier's.

Hector AI, our artificial intelligence platform, also steps in at this stage: where attributes arrive mixed into the description, extracting them and checking their consistency is the work described on the page about product pages generated with Hector AI.

Publishing rules: what goes online, and from which source

Duplicate products and choice of source

When the same EAN appears in the price lists of several suppliers, the ERP keeps a single product page with several linked offers. The active source depends on the purchase price, on actual stock availability and on a per-supplier priority that we can set. If the chosen supplier drops to zero, the offer switches to the next one and the online price follows: the customer always sees a single page, with consistent price and availability.

Exclusions and minimum requirements

A product without a description, image or category is not published until it meets the minimum requirements. The same applies to product families excluded by commercial choice, to items being permanently discontinued by every supplier, and to anomalous prices: better one product page fewer than a wrong price on the storefront.

Stock availability and prices: an update has to reach the search index

Prices and stock availability are the data that change most often and cost the most when they are wrong. The ERP pushes them to WooCommerce with a differential flow: only what has changed is written, because rewriting the entire catalogue every time would bring the database to its knees. A price updated in the database is not yet updated for the site's visitors: the page may be served by Varnish, the product object may sit in Redis and the search result comes from Elasticsearch. Every update has to propagate along the whole chain — targeted page invalidation, object cache refresh, product reindexing — otherwise search suggests a price the cart does not recognize. How the index is organized, and why we abandoned WooCommerce's native queries, is explained on the page about Elasticsearch for e-commerce search.

Error handling: what happens when something does not add up

With several suppliers and hundreds of thousands of rows, something fails to add up on every cycle. The question is where it gets stopped and how it is made visible.

Pipeline stage	What it does	What can go wrong	How we handle it
Ingestion	Fetches and reads each supplier's price list	Missing or truncated file, changed layout	Completeness check; the invalid price list is ignored and the last good one stays in use; automatic retry and alert
Normalization	Brings codes, units, prices and attributes into the internal format	Incorrect EANs, ambiguous units, prices out of range	Row discarded with the reason logged; anomalous deviations quarantined pending review
Publishing	Applies the rules on what to publish and from which source	Missing minimum requirements, conflict between sources	The product page stays unpublished or keeps the last valid offer; the case goes into the report
Synchronization	Writes the differences to WooCommerce	Write errors, timeouts, database load	Batches of controlled size, resumption from the point of interruption, a log entry for every write
Propagation	Invalidates Varnish and Redis, reindexes in Elasticsearch	Cache not invalidated, index not updated	Reindexing queue with confirmation; consistency check between database and index

Quarantine deserves a word more. A price that deviates abnormally from the previous one may be a supplier error or a genuine promotion; the system does not decide on its own: the row is isolated, the product keeps its last valid data and an operator examines it. The log records every decision, so an odd price reported by a customer can be traced back to the source and to the cycle that produced it.

Scheduling, load and marketplaces

Synchronizations do not all run at the same pace: stock availability changes far more often than descriptive data, and each supplier publishes on its own calendar, so scheduling follows the timing of the sources. Heavy cycles, such as a full re-read of a price list, go into the low-traffic windows and are split into batches: the infrastructure withstands peaks only if we are not the ones causing them.

The same ERP also feeds the marketplaces. Publishing on Amazon starts from the normalized catalogue, with the channel's own exclusion rules, and a proprietary repricer recalculates the price taking into account the target margin, the fees and the actual stock availability at the suppliers. When the source changes or stock runs out, the marketplace price follows, and an unavailable product is withdrawn before it generates an order we cannot fulfil. This part of the system is in production and we keep evolving it in our work on AI for logistics and operations management.