Schema Evolution - CI/CD Glossary Definition
Schema evolution is changing a datasets structure over time while keeping backward compatibility for existing readers.
Schema evolution is the controlled change of a datasets structure over time (adding, renaming, or retyping columns) while keeping existing readers working. Additive changes are safe; removals and type changes are breaking.
Serialization frameworks and table formats provide rules for which schema changes are safe, so pipelines evolve without breaking consumers.
Safe vs breaking
Adding a nullable column is backward compatible. Dropping a column, renaming, or narrowing a type is breaking and should fail a data-contract check in CI.
Related guides
Data Contract - CI/CD Glossary DefinitionData Contract: A data contract is an explicit, versioned agreement between a data producer and consumer speci…
Data Validation - CI/CD Glossary DefinitionData Validation: Data validation is the automated checking of data against rules (types, ranges, nullability,…
Data Lineage - CI/CD Glossary DefinitionData Lineage: Data lineage is the traceable map of how data flows from sources through transformations to fin…