Custom Changeset PublisherΒΆ

This section provides guidance on implementing your own custom Changeset Publisher. The Discovery Data Tool provides an open source implementation that you can reference, extend, or use directly.

The changeset publisher returns a valid changeset for all records that have changed since it was last run. This means that it is a stateful service. Calls the changeset publisher pass in a profile name, this allows the implementation to maintain it’s state.

For example, the engine could be configured with a feed URL of:

http://example.com/admin/changesets.php?profile=production

A first call to this URL would result in a changeset for the entire dataset. Subsequent calls would result in a changeset containing only records that have changed since the last changeset was pulled.

We recommend that you create new profiles on demand. This means that any profile that hasn’t previously been pulled would result in a changeset containing the entire changeset. This can greatly simplify configuring multiple engines for automatic failover support.

A typical implementation would use a table similar to the following to track it’s state.

create table ChangesetProfile (
  name varchar(20) not null,
  lastRun timestamp
)

If you need to force the engine to pull a clean data set, then you could set the lastRun column to null for your particular profile.

To determine which records have changed, we recommend the following procedure.

  1. Look up the lastRun timestamp for the current profile as start.
  2. If start is null then set the following HTTP header:
X-t11e-type: snapshot
  1. Get the current database time as end.
  2. If start is null then perform the following query for each view:
select * from MyView
where lastModified >= start and lastModified < end
  1. If start is not null then perform the following query for each view:
select * from MyView
where lastModified < end
  1. Stream out the changeset <set-item> elements.
  2. Repeat steps 4 through 6 against the deleted record views, streaming out <remove-item> elements instead.
  3. On success save end as into the lastRun column for the current profile.