Understanding Logical Data Entity Statuses

When a logical data entity is created during harvest, the status of the logical data entity is Active.

Logical data entities are a created from the actual names of the files in the Object Storage buckets, the filename pattern expression, and the filename patterns assigned to the data assets. When there is a change to the Object Storage files, filename pattern expression, or the associated filename patterns, the status of the logical data entity can change.

  • Object Storage: If your Object Storage buckets have more or less files from the last harvest, data catalog reflects the changes on reharvest. New files are added to the logical data entity and deleted files are removed from the logical data entity.
  • Filename pattern expression: When you modify an expression in a filename pattern, no certainty that the logical data entities created on reharvest are same as the logical data entities created during the previous harvest. To indicate this state of uncertainty, Data Catalog sets the status of the logical data entities harvested using the expression to Inactive. During reharvest, if same logical data entities are derived, Data Catalog sets their status to Active again. Logical data entities from the previous harvest that weren't derived during reharvest remain Inactive. You need to explicitly delete them as needed.
  • Filename patterns: Consider you assign a new filename pattern to a data asset or unassign any existing filename patterns from a data asset. no certainty that the logical data entities created on reharvest are same as the logical data entities created during the previous harvest. To indicate this state of uncertainty, Data Catalog sets the status of the logical data entities harvested using the filename patterns to Inactive. During reharvest, if same logical data entities are derived, Data Catalog sets their status to Active again. Logical data entities from the previous harvest that weren't derived during reharvest remain Inactive. You need to explicitly delete them as needed.