mlm_insights.core.transformers package

Subpackages

Submodules

mlm_insights.core.transformers.conditional_feature_transformer module

class mlm_insights.core.transformers.conditional_feature_transformer.ConditionalFeatureMetadata(feature_metadata: mlm_insights.core.features.feature.FeatureMetadata, expression: mlm_insights.core.transformers.expression_evaluator.Expression)

Bases: object

expression: Expression
feature_metadata: FeatureMetadata
class mlm_insights.core.transformers.conditional_feature_transformer.ConditionalFeatureTransformer(conditional_features: List[ConditionalFeatureMetadata])

Bases: Transformer

Base Class that defines a ConditionalFeatureTransformer.

This evaluates conditional feature expressions and creates new features/columns on the input dataframe.

classmethod create(config: Dict[str, Any]) → ConditionalFeatureTransformer

Factory Method to create a ConditionalFeatureTransformer.

The conditional feature metadata will be available in config.

Returns

ConditionalFeatureTransformer

An Instance of ConditionalFeatureTransformer.

Examples

conditional_feature_metadata_no_column_type = ConditionalFeatureMetadata(
feature_metadata=FeatureMetadata(
    feature_name="feature_1",
    feature_type=FeatureType(
        data_type=DataType.INTEGER,
        variable_type=VariableType.DISCRETE
    )
),
expression=Expression("df['test_column']%2==0", ExpressionType.python))

conditional_feature_metadata_with_column_type = ConditionalFeatureMetadata(
    feature_metadata=FeatureMetadata(
        feature_name="feature_2",
        feature_type=FeatureType(
            data_type=DataType.INTEGER,
            variable_type=VariableType.DISCRETE,
            column_type=ColumnType.TARGET
        )
    ),
    expression=Expression("df['test_column']%2==0", ExpressionType.python))

cf_transformer = ConditionalFeatureTransformer(conditional_features=[conditional_feature_metadata_no_column_type,
                                                                     conditional_feature_metadata_with_column_type])
output_schema = cf_transformer.get_output_schema(input_schema=pyarrow.schema(fields=[]))
assert output_schema is not None
column_type_1 = output_schema.field_by_name("feature_1").metadata.get(b'column_type', None)
assert column_type_1 is not None
assert column_type_1.decode("utf-8") == ColumnType.INPUT.name

column_type_2 = output_schema.field_by_name("feature_2").metadata.get(b'column_type', None)
assert column_type_2 is not None
assert column_type_2.decode("utf-8") == ColumnType.TARGET.name
get_output_schema(input_schema: Schema, **kwargs: Any) → Schema

Generates a new schema after transforming conditional features.

Parameters

input_schema: pa.Schema

Schema of the input data frame

Returns

output_schema: pa.Schema

Feature schema for the transformed dataset after transforming Conditional Features

transform(data_frame: DataFrame, **kwargs: Any) → DataFrame

Evaluates Conditional Feature Expressions and create new Features applying Conditional Transformer and append it to input dataframe.

Returns

pandas DataFrame: DataFrame containing all the features after applying Conditional Transformer

validate() → List[ComponentValidationResult]

Validates Conditional Feature Expressions.

Returns

List:

list of ComponentValidationResult

mlm_insights.core.transformers.expression_evaluator module

class mlm_insights.core.transformers.expression_evaluator.Expression(value: str, type: mlm_insights.core.transformers.expression_evaluator.ExpressionType)

Bases: object

type: ExpressionType
value: str
class mlm_insights.core.transformers.expression_evaluator.ExpressionEvaluator

Bases: ABC

evaluate(expression: Expression) → Any
validate(expression: Expression) → ValidationResult
class mlm_insights.core.transformers.expression_evaluator.ExpressionType(value)

Bases: Enum

An enumeration.

python = 'python'
class mlm_insights.core.transformers.expression_evaluator.ValidationResult(success: bool, expression: str = '', message: str = '')

Bases: object

expression: str = ''
message: str = ''
success: bool