mlm_insights.tests package

Subpackages

Submodules

mlm_insights.tests.constants module

class mlm_insights.tests.constants.GroupByKey(value)

Bases: Enum

Enumeration for various grouping keys for grouping Test Results

FEATURE_TAG_KEY = 'feature_name'
TEST_STATUS_KEY = 'test_status'
TEST_TYPE_TAG_KEY = 'test_type'

mlm_insights.tests.framework_tests_enum module

class mlm_insights.tests.framework_tests_enum.FrameworkTestsEnum(value)

Bases: Enum

Insights-provided Tests and Test Suites.

TestDeviation = <class 'mlm_insights.tests.test_types.predicate_based_tests.test_deviation.TestDeviation'>
TestEqual = <class 'mlm_insights.tests.test_types.predicate_based_tests.test_equal.TestEqual'>
TestGreaterThan = <class 'mlm_insights.tests.test_types.predicate_based_tests.test_greater_than.TestGreaterThan'>
TestIsBetween = <class 'mlm_insights.tests.test_types.predicate_based_tests.test_is_between.TestIsBetween'>
TestIsComplete = <class 'mlm_insights.tests.test_types.metric_based_tests.test_is_complete.TestIsComplete'>
TestIsMatchingInferenceType = <class 'mlm_insights.tests.test_types.metric_based_tests.test_is_matching_inference_type.TestIsMatchingInferenceType'>
TestIsNegative = <class 'mlm_insights.tests.test_types.metric_based_tests.test_is_negative.TestIsNegative'>
TestIsNonZero = <class 'mlm_insights.tests.test_types.metric_based_tests.test_is_non_zero.TestIsNonZero'>
TestIsPositive = <class 'mlm_insights.tests.test_types.metric_based_tests.test_is_positive.TestIsPositive'>
TestLessThan = <class 'mlm_insights.tests.test_types.predicate_based_tests.test_less_than.TestLessThan'>
TestStringEquals = <class 'mlm_insights.tests.test_types.predicate_based_tests.test_string_equals.TestStringEquals'>

mlm_insights.tests.profile_source module

class mlm_insights.tests.profile_source.ProfileSource(value)

Bases: Enum

An enumeration describing the source type of the Insights Profile

CURRENT = 1
REFERENCE = 2

mlm_insights.tests.test_result module

class mlm_insights.tests.test_result.TestAssertionInfo(expected: Any | None = None, actual: Any | None = None)

Bases: object

Describes the expected and actual values for a test. For eg: when running a TestGreaterThan for a Mean metric with a threshold value of 200, expected contains the threshold value (200) and actual contains the value of Mean metric.

actual: Any = None
expected: Any = None
class mlm_insights.tests.test_result.TestError(description: str = '', has_error: bool = False)

Bases: object

Describes test error encountered during test execution.

description: str = ''
has_error: bool = False
class mlm_insights.tests.test_result.TestResult(name: str, description: str, status: ~mlm_insights.tests.test_result.TestStatus, test_config: ~typing.Dict[str, ~typing.Any] = <factory>, test_assertion_info: ~mlm_insights.tests.test_result.TestAssertionInfo = <factory>, error: ~mlm_insights.tests.test_result.TestError = <factory>, user_defined_tags: ~typing.Dict[str, str] = <factory>, system_tags: ~typing.Dict[str, str] = <factory>)

Bases: object

Class describing result in standard format returned by each test

description: str
Each test when executed produces a status which is one of the following: PASSED, FAILED, ERROR

  • When test passes a given condition, status is set to PASSED

  • When test fails a given condition, status is set to FAILED

  • When test exeuction encounters an error, status is set to ERROR

error: TestError

User-defined tags specified as a key value pair where both key and value must string

name: str

Test description in a structured format

status: TestStatus
system_tags: Dict[str, str]
test_assertion_info: TestAssertionInfo
test_config: Dict[str, Any]

Each test returns the expected and actual information which helps in understanding why a particular passed/failed

to_dict() → Dict[str, Any]
user_defined_tags: Dict[str, str]

System-defined tags specified as a key value pair where both key and value must string

class mlm_insights.tests.test_result.TestStatus(value)

Bases: Enum

Enum describing the various test status.

ERROR = 2
FAILED = 1
PASSED = 0
classmethod get_status(status_bool: bool) → TestStatus

mlm_insights.tests.test_results module

class mlm_insights.tests.test_results.TestResults(test_summary: ~mlm_insights.tests.test_summary.TestSummary, test_results: ~typing.List[~mlm_insights.tests.test_result.TestResult], user_defined_tags: ~typing.Dict[str, str] = <factory>)

Bases: object

Class describing the results of test evaluation. Consists of 2 high-level properties:

  • Test Summary

  • List of test result for each configured test

Provides methods to group tests by following grouping keys: feature, test type, test status and tags

group_tests_by(group_by: GroupByKey) → Dict[str, List[TestResult]]

Group the test results by a key. Supported groupby keys are: feature_name, test_type, test_status

test_results: List[TestResult]
test_summary: TestSummary
to_json() → Dict[str, Any]

Create a json View of Insights TestResults.

Returns

Dict : Dictionary of Test Results

user_defined_tags: Dict[str, str]

mlm_insights.tests.test_summary module

class mlm_insights.tests.test_summary.TestSummary(total_tests: int = 0, passed_tests: int = 0, failed_tests: int = 0, error_test: int = 0)

Bases: object

Test Summary returns the following information about the executed tests.

  • Count of tests executed

  • Count of passed tests

  • Count of failed tests

  • Count of error tests

    • Tests error out when the test validation fails or error is encountered during test execution

error_test: int = 0
failed_tests: int = 0
passed_tests: int = 0
total_tests: int = 0

mlm_insights.tests.test_validation_result module

class mlm_insights.tests.test_validation_result.TestValidationResult(valid: bool, errors: ~typing.List[str] = <factory>)

Bases: object

TestValidationResult tells if a Validation done by an Insight Test is valid or not. Insight Test must set valid = True, if validation succeeded. If validation fails, it should set valid = False and additionally it also set the list of errors

errors: List[str]
is_valid() → bool

Returns

Return if this validation succeeded of Failed.

valid: bool
class mlm_insights.tests.test_validation_result.TestValidationResults

Bases: object

Wrapper class around a list of TestValidationResult

add_validation_result(validation_result: TestValidationResult) → TestValidationResults
get_results() → List[TestValidationResult]
has_errors() → bool
validation_results: List[TestValidationResult]

Module contents