Synchronizing Data with “changedSince”

A number of Erply API calls have an input parameter “changedSince”.

You can use this parameter to efficiently synchronize data to another application – by only pulling data that has been added, changed or deleted.

Here is a sample walkthrough for products. Other kinds of items (customers, sales documents, gift cards, etc.) can be synchronized in a similar manner.

First step: download full data set

  1. First, download full product list with API “getProducts“. This can take time, and usually requires multiple calls, since API returns a fixed number of items at a time. Set recordsOnPage = 100 and download the data in “pages”: pageNo = 1 for the first call, pageNo = 2 for the second, and so on.
  2. API has an hourly request limit of 1000 calls. If you have a lot of products (hundreds of thousands), you might not be able to download the whole list in one hour. API will start returning error code 1002 once the limit has been reached. Your script should be able to detect that case, inform the operator, and save its progress so that work could be resumed next hour.
  3. For an advanced optimization, you may utilize “bulk calls“, which allows to download data at 10x regular rate.
  4. In each API response, there is a field “requestUnixTime” — a timestamp. Take note of the value that server returned when you issued your FIRST “getProduct” call. If synchronization completes successfully, keep that value. This is the time when last successful synchronization was started.

Incremental synchronization

To bring your local database up to date, you need to retrieve items that have been added, changed, and deleted meanwhile. These are the steps; you can set the script to run at regular intervals.

  1. To retrieve added and changed items, call API “getProducts” and send the last synchronization timestamp as input parameter “changedSince”. API returns items that have been added or changed since that moment of time. Number of returned items may exceed 100, so be ready to use paging if necessary (see step 1 in previous section).
  2. When issuing the FIRST getProducts call, take again note of the “requestUnixTime” timestamp in response header and set it aside.
  3. For each item that getProducts returns, check if you already have an item with that ID in your local database. If yes, update it. If not, add it as a new item.
  4. To retrieve DELETED products, use API call “getUserOperationsLog“: tableName = “products”, addedFrom = last synchronization timestamp
  5. API returns a list of product IDs (field “itemID”). Delete all these items from your local database.
  6. If synchronization completes successfully, update the timestamp of last successful synchronization: take the value you retrieved in step 2 and store it in database. Use it for your next synchronization; then update it again after synchronization completes successfully; etc.

Tips and Recommendations

  • Use only server’s timestamps — the ones returned in every response header (field “requestUnixTime”). Your clock may be different from server clock, so if you stamp the synchronization with a locally generated timestamp, you’ll risk missing some updates.
  • Product’s timestamp only changes when product card is updated in Erply. On the other hand, timestamp DOES NOT CHANGE when product’s inventory quantity changes, eg. as a result of a sale. To synchronize inventory quantities, use API “getProductStock” instead.