mlm_insights.mlm_dask.execution_engine package
Submodules
mlm_insights.mlm_dask.execution_engine.dask_engine module
- class mlm_insights.mlm_dask.execution_engine.dask_engine.DaskEngine
Bases:
ExecutionEngine
Class for Dask execution engine functionality.
- classmethod create_client(engine_detail: EngineDetail, **kwargs: Any) Client
Method to create dask engine specific client.
Parameters
- engine_detail: EngineDetail
Engine Detail object can hold Any type like ‘dask’, ‘spark’ as string
Returns
Dask Engine LocalCluster Client
- get_schema_provider(input_schema: Dict[str, FeatureType]) DaskSchemaProvider
Method to convert dask engine specific schema using user schema.
Parameters
- input_schema: Dict[str, FeatureType]
key-value pair
key: attribute name from data set
value: attribute data type and variable type
Returns
Dask engine specific schema