Syncing Data

Best practices for syncing data between you and Merge
Overview

When syncing data with Merge, we recommend a combination of webhooks and polling.

  1. Save your users' account token
  2. Get the account token from the linking process for an embedded Merge Link. Learn more in our embedded Merge Link guide.

    You can also use the Linked Account linked webhook to get the account token. See the example payload on the right and learn how to configure Merge Webhooks in our guide.

    To authenticate your API requests to Merge, save your users' account token in your database. You will need the account token to poll for data in step 4.

  3. Sync data when Merge emits a sync notification webhook
  4. We recommend using the Linked Account synced webhooks to manage sync activities at scale. Whenever you receive a sync notification webhook for a Linked Account, start pulling data and kick off the logic in step 3.

    Important fields
    hook.event

    The event type that triggered the webhook.

    See our webhooks guide for more information.

    linked_account.idThe ID of the associated Linked Account.
    data.sync_status

    Handle edge cases when last_sync_result is FAILED or PARTIALLY SYNCED.

    See our Help Center article on sync statuses

  5. Create functions for efficiently syncing data
  6. Store the timestamp of when you last started pulling data from Merge as modified_after. Use this timestamp in subsequent API requests to pull updates from Merge since your last sync.

    Use the expand parameter to pull multiple models that are related to each other instead of making multiple pulls for related information.

    Query Parameters

    GET
    /{endpoint}/?modified_after={timestamp}

    Only pull data that has been changed or created since your last sync.

    For example, you can ask for modified_after=2021-03-30T20:44:18, and only pull items that are new or changed.


    GET
    /{endpoint}/?expand={common_model_name}

    Pull related model information with a single API request.

    For example, if you are querying for candidates and also want details about associated applications, you can expand=applications, and Merge will return the actual application objects instead of just the application_id.

  7. Sync periodically and poll using /sync-status endpoint
  8. Make a request to our /sync-status endpoint, which returns an array of syncing statuses for all models in a category. See API reference to learn more.

    For the first ping:

    • If status is PARTIALLY SYNCED or DONE , go ahead and retrieve data

    • If status is SYNCING , retrieve the last_sync_start and continue pinging

    For the next pings:

    • If status is PARTIALLY SYNCED or DONE , andlast_sync_start is different from the initial last_sync_start , go ahead and retrieve data

    • If status is SYNCING and last_sync_start is the same as the initally retrieved last_sync_start , continue pinging