mlm_insights.core.data_sources.utils package¶
Submodules¶
mlm_insights.core.data_sources.utils.constants module¶
mlm_insights.core.data_sources.utils.data_location_utils module¶
- class mlm_insights.core.data_sources.utils.data_location_utils.DataLocationUtils¶
Bases:
object- static create_data_location(location: str, offset: int, date_range: Dict[str, str], file_type: str) List[str]¶
- static create_date_range_dict_from_offset(offset: int) Dict[str, Any]¶
- static create_location_arr_from_date_range(location: str, date_range: Dict[str, str], file_type: str) List[str]¶
- static now() datetime¶
This method returns datetime now so that now() can be mocked in testing
- static validate_bucket(bucket_name: str, namespace: str, object_prefix: str) None¶
- static validate_data_location(offset: int, date_range: Dict[Any, Any], date_format: str, file_type: str) None¶
- static validate_date_text(date_text: str, date_format: str) None¶
mlm_insights.core.data_sources.utils.data_source_utils module¶
- mlm_insights.core.data_sources.utils.data_source_utils.get_data_source(file_path: List[str] | str, **kwargs: Any) DataSource¶
Description : method to identify the file system
Parameters¶
- file_path :
file_path of the file for underlying file system. For oci it starts with oci:
Returns¶
- data_source: DataSource
instance of DataSource