Documentation and data governance
Documentation and data governance are key to access trusted, secure, quality assured data that can be understood and used correctly.
Why it matters
- An organization's ability to proactively curate and leverage data to make better and more informed decisions, depends on proper documentation and data governance
- Documentation is required to understand what a specific data point or a number in a report means. Examples include where the data is coming from, who can be asked about it, and how it can be used to solve a business problem.
- Without automation, documentation is often set to lower priority and seldom maintained properly, which puts people in risk of misusing or misinterpreting data
- Data governance defines how people and applications can interact with data and is crucial to safeguard both people, applications and data while staying within regulatory requirements
- In the future, governance will likely also set limits for responsible AI
How it works
- Xpert BI has a metadata engine with an end-to-end dependency graph for all tables and columns, this enables complete column level data lineage and documentation
- Documentation is automatically generated always reflecting the SQL code running in production
- Project managers will get a list of objects not described by the data team and once an object is updated, it is live in the documentation
- An important Data Governance excercise is to define ownership to data and reports, this can be done using tags down to separate columns, it later helps users to answer "who can I ask?" or business analysts to answer "who shall I involve?"
- Data governance work is also supported with data lineage graphs with full dependencies, seeing where data is coming from, where it is used, tracking sensitive data etc
- DataOps is also related to data governance with test automation for data quality monitoring, and source control management for better collaboration and reduced operational risks