Warehouse sync · How it works

One small service at the site. Centralized data in the cloud.

Database changes flow one way: out of each site, into a central store your application reads. Nothing at the site is replaced. Nothing reaches in.

  • SQL Server
  • MongoDB
  • Postgres
  • Outbound only
  • Up to date in minutes
Customer sites
McDonough, GAAtlanta area · Eastern timeSQL Server · change trackingredfly
Lancaster, TXDallas area · Central timeSQL Server · change trackingredfly
Fontana, CALos Angeles area · Pacific timeSQL Server · change trackingredfly
  • Only changes
  • Compressed
  • Encrypted per site
  • Offline support
Your cloud
Central StoreLow-cost storage. Every row stamped with its site. Current to within minutes.
Your applicationDashboards, support tools, analytics.
Who uses it
Your customers
Your support desk
Your product team
Only changes travelA first full copy, then rows added, edited or deleted.
Queued at the siteOn disk, in ranked order, until the cloud confirms.
Per-site encryptionCompressed and encrypted with a key unique to the site.
One central storeEvery row stamped with its site and arrival time.
What runs where

Three places, one direction.

At the site

A small service beside the site database reads changes, queues them on disk, and sends them out.

On the wire

Batches of changes, compressed, encrypted with a per-site key, over outbound connections only.

In the cloud

A receiver writes each site into low-cost document storage. Your application reads from there.

What a site needs

Less than you would expect.

We set it up together with your contact.

  • A server, VM or PC in the site's network with access to the database and outbound internet.
  • Specify the prioritized list of tables to be synced.
  • Specify the filters to prioritize rows.
  • Outbound internet only. No port is opened to the internet.
  • One technical contact for the first install.
The data path

From a row changing to your application seeing it.

  1. 1

    A first full copy, table by table

    Sent once. Everything after that is changes only.

  2. 2

    Changes read on a short interval

    Only the rows and columns that changed since the last look.

  3. 3

    Queued on the site's disk

    In ranked order. Removed only once the cloud confirms receipt.

  4. 4

    Sent in batches, compressed and encrypted

    Every few tens of seconds, over an outbound connection.

  5. 5

    Applied centrally within a couple of minutes

    At-least-once delivery, so nothing is lost.

Reliability

On a bad day.

The link drops

Changes keep queueing on disk and go when it is back.

Something restarts

Each table resumes from the last change it sent.

Cut off for days

Resumes from where it left off when the site is back. Nothing is lost.

Rows edited and deleted

Carried through, so the copy never drifts from the truth.

A site goes silent

It reports its health; a stalled site is noticed, not discovered in a report.

The site gets busy

Dynamic throttling to preserve operational database performance under load.

Keeping the site first

The operation never waits for the sync.

  • Dynamic throttling under load. Operational database performance comes first.
  • You rank the tables. Important ones first; large backlogs trickle in the background.
  • Light reads. After the first copy, only what changed, in a consistent snapshot.
  • No reporting on the site. Your application reads the central copy.
The central store

The current state of every site, in one place.

Current state, not a log

Edited at the site, edited centrally. Deleted at the site, deleted centrally.

Every row stamped

Site and arrival time on every row; each site in its own space.

Spend hundreds, not thousands

Document storage, not a database engine per site. Read it with your own tools.

We keep the data flowing. Your team builds what your customers see.

Boundaries

What it is not.

  • One-way. Nothing is written back to the site.
  • Database tables only. Not files, video, device streams or remote control.
  • Within a couple of minutes, not instantly.
  • Not a local read copy. Site applications keep reading their own database.
  • Not an event history. The store holds what each table looks like now.
  • Each site needs its own setup. A person drives the rollout.
Who runs what

Clear from the first site.

redfly

Installs and operates the site service, watches its health, handles backfill and recovery.

The cloud account

Your own account (preferred), fully managed by redfly, or onsite. No proprietary cloud services.

Your team

Chooses and ranks the tables, owns the application, provides one contact per site.

Design-partner contract with an architecture review, then an annual or monthly subscription. See how pricing works

Ready when you are

Tell us about one site.

The database it runs, the link it has, and the application that needs the data.