FU

Wednesday, May 5th, 2021 10:33 AM

Data As Code - Tying Metadata to Deployment

In addition to the approach of harvesting data and looking to determine the data meaning through guided stewardship (discovery or ‘run time’ metadata gathering), we are interested in approaches to declaring data meaning as part of code or data deployment (declarative or ‘design time’ metadata gathering).
Possible approaches would include:

API deployments built using OpenAPI specifications can contain YAML sections describing the data content available through the API, which can be integrated into Collibra as part of the deployment pipeline.
Teams using model driven development approaches can integrate Collibra with data modelling tools such as Enterprise Architect and have publication workflows synchronised with application deployment pipelines.

We also see the potential for a kind of hybrid approach where teams use the data harvesting approaches in QA or Pre-Production environments to validate or pick up changed metadata mappings and then these also get published into Collibra upon deployment.

Would be interested in any experiences of these approaches, or other ways to improve the reliability of the metadata in Collibra by tying it to the software development lifecycle.

1.2K Messages

3 years ago

There were two great presentations at the last France User Meetup, by CMA-CGM and Adeo on that approach.

The slides and recording are available here: Data Citizens: France - :earth_americas:Regional Conversations - The Data Citizens Community (collibra.com)

@olivier.vanduynslaeger.adeo.com, @maximilien.cote.capgemini.com: interested to add to it?

5 Messages

3 years ago

Great slides - thanks a million for pointing us at them. That is really helpful and is very similar to the design we have in our heads. Can I ask if the schema references in the YAML reference the data model (Data Entity/Data Attribute) or do they generate a separate physical data model?

Loading...