Strake (Simple Transactional Rake)

Posted by Joeri Samson on Dec 03, 2009

Strake is a tool we developed internally here at XAOP, for certain specific tasks. Strake basically eases the development cycle in certain use-cases

In particular we use it when we have a project that mainly consists of transforming data from one representation to another (e.g. when we migrate documents from a legacy Documentum system to a newer Documentum system with a different layout). In that case it is often necessary to gradually apply changes to the data. To do this in a repeatable way we write Strake tasks which work very much like database migrations in that they are ordered.

The useful feature added by Strake is that it keeps snapshots from the state of the database before the strake task was run. This makes it possible to restart the process of running strake tasks from any point in time (if for example a mistake was discovered).

The normal way we go about this is to receive mappings provided by the user, and to translate these to code. Then after some of the most glaring issues are worked out (like, documents that have to be mapped in contradictory ways), we make the data (or a subset) available for the user to review and analyze all the transformations through a webinterface This will usually bring out some issues that need to be resolved, (for example attributes that need to be filled out in the new system, but that are empty, values that do not fit in the target system, ...), some of these can be solved for individual documents, others are issues with the transformation process and lead to an updated mapping.

Once the data is in the analyzer we can't use Strake anymore for it's primary purpose, because users can edit the data in the analyzer, so if we just rerun the strake tasks from a certain point in time information will be lost. Personally I still find it useful that Strake keeps taking snapshots at this stage, because this provides automatic back-ups of the data at the points of the process where things are most likely to fail. It is of course also still useful during the short phases of development that take place outside the analyser, where our changes to the mapping are tested, so that it's possible to work on more than one transformation at the same time.

This process is repeated until no flaws with the migration are found anymore, at that point the transformed data will also be loaded in the target system.

Strake was released for the public some time ago, it's open-source so you can add support for more databases (currently strake only works with MySQL).