3CpO is the name we give to the ODE framework.  Its an acronym for three C’s plus an O.  These are:

  • Configuration
  • Code Generation
  • Code Execution
  • Orchestration

ODE 3CPO

Config

Config is the relational data model we have built that holds all the configuration required to make ODE run.

The goal is to allow you to create and populate a full Data Vault model, purely by adding config to ODE.

This includes definitions, for example

  • Raw Vault Hub, Sat and Links
  • Business Vault Hub, Sats and Links
  • Same as Links
  • Presentation views to be deployed (Stars Schemas, Denormalised tables etc)

The confif also includes options on how you populate the data vault, for example:

  • Virtualise the presentation views or instantiated them.
  • End date records, create tombstone records or do both

Config is the heart of ODE, without it there is nothing.

Code Generation

Code Gen is the code that builds Code Exec.

It reads the config and generates the code that is needed to create the structures and move the data.
(of course if you select the the virtualised options it will create views that see the data).

Code Gen might come in multiple flavours, so you can deploy it on the technology you already have.  Right now it is only develop and tested on Microsoft SQL Server.

Code Execution

Code Exec is the code that runs to create the structures or move the data.
(of course if you select the the virtualised options it will be views that see the data).

At the moment Code Exec is persisted as Stored Processes.  Code Exec is maintained as part of the core of the ODE engine.

Any business rules or transformatons you want to apply to data as you load it are written by you, stored in Stored Procs and register in config.  This allows the orchestration engine to add them to the load manifest.

We have a dream of moving the code into the database as text within tables.  One reason is we can use that same design pattern to store the custom business/transformtation rules there as well.   Whole raft of other technical benefits for us doing this, maybe one day.

Code Exec also comes in the same multiple flavours as Code Gen, so right now SQL Server only.  Although as its SQL statements its pretty transportable in theory.

Orchestration

Orchestration is the engine that runs the Code Exec to actually load and trasnform the data.

Manifest Mode

When you orchestrate a load, ODE looks at the config and effectively creates a manifest that it then executes.  There is a way cool multi stream capability whihc optimises your loads and manages failures.

Code Deployment

Code Deployment allows you to promote the cofig across your different environments (i.e. Dev > Test > Prod)

Execution

The execution engine that actually tells the orchestartion engine to exectute te manifest  is currently over to you.

It can be as simple as Cron, a external scdueling or workflow capability or the use of an ETL engine such as SSIS.  We use SQL Agent in all our deployments right now.