mlm_insights.mlm_dask.interfaces package¶
Submodules¶
mlm_insights.mlm_dask.interfaces.dask_data_reader module¶
- class mlm_insights.mlm_dask.interfaces.dask_data_reader.DaskDataReader(name: str, file_path: List[str] | str = '', data_source: DataSource | None = None, **kwargs: Any)¶
Bases:
DataReaderData Reader base class for Dask execution engine. Dask-based readers must derive from this class and implement the abstract methods.
- abstract read(schema_provider: SchemaProvider, **kwargs: Any) Any¶
Read the data from the data-source and return engine specific data frame.
Parameters¶
- schema_providerSchemaProvider
Engine specific schema provider. Use it to call get_engine_schema() to get engine specific schema to pass to the respective read methods
- kwargs:
Key-value pair for dynamic arguments
Returns¶
- dataframe: Any
Engine specific DataFrame