driftai package¶

Submodules¶

driftai.approach module¶

class driftai.approach.Approach(project, name, subdataset=None, path=None, creation_date=None)[source]¶

Bases: driftai.db.persistent.Persistent

Responsible of containg approach information as well as generate approach tree structure

Parameters:	project (Project) – DriftAI Project name (str) – Approach name subdataset (Subdataset) – DriftAI Subdataset which contains the training instances path (str, optional) – Path to store the approch. Default value is <project.path>/approaches/<name> creation_date (str, optional) – Date of the approach creation. Should not be set manually

static collection()[source]¶

Get table containing approaches

Returns:
Return type:	TinyDB instance

create_structure()[source]¶: Generate approach directory structure

get_info()[source]¶

Get the info to serialize a Dataset instance

Returns:

Dictionariy containing a Approach object summary:

{
    "id": <unique identifier>,
    "project": <project containing the approach>,
    "subdataset": <subdataset which approach will run against>,
    "name": <approach name>,
    "path": <approach file system location>,
    "runs": <runs which runnable approach will execute>,
    "creation_date": <approach creation date>
}

Return type: dict

get_last_run_date()[source]¶: Get the date of the last run

id¶

Get the unique identifier of the Persistent instance

Returns:	Unique identifier
Return type:	str

classmethod load_from_data(data)[source]¶

Load approach from JSON coming from TinyDB

Parameters:	data (dict) – Dict containing the approach instance summary
Returns:	Approach instance generated from JSON data
Return type:	Approach

status¶

Get the status of approach runs

Returns:	Dictionary containing the done and left runs … { "done": <are ther pendent runs?>, "total_runs": <all runs>, "done_runs": <done runs>, "progres_bar": <string containing the progress bar> }
Return type:	dict

class driftai.approach.RunnableApproach(**kwargs)[source]¶

Bases: abc.ABC

Object responsible to handle an approach execution

Inherit from this class in order to create your own approach and run it

inference(data)[source]¶

Define the inference logic here

Returns:	Containing the predicted labels
Return type:	pandas.Series or numpy.array

learn(parameters, data)[source]¶

Define the train logic for the model here

Returns:	The trained model
Return type:	any

parameters¶

Define your parameters range here

Returns:	Parameters used to generate the runs
Return type:	list of AbstractParameter

run(resume=False)[source]¶

Run the approach

Parameters:	resume (bool) – If resume = True only pending runs will be executed, otherwise run will be generated and executed

driftai.exceptions module¶

exception driftai.exceptions.OptAppDatasetInfoFileDoesNotExistException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppFileDatasourceNotCompatibeException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppInstanceExistsException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppInvalidStructureException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppMethodNotImplementedYetException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppProjectDirNotExistsException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppProjectElementNotExistsException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppProjectFileNotExistsException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppProjectLoadPathIsNotDirException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppProjectNameExistsException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppProjectWrongInfoFileStructureException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppRunFileDoesNotExistException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppRunFileWrongStructureException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppSubDatasetInfoFileDoesNotExistException[source]¶: Bases: Exception

exception driftai.exceptions.OptAppSubDatasetInfoFileWrongStructureException[source]¶: Bases: Exception

driftai.project module¶

Optimise approaches to solve machine learning problems.

TODO: Change random seed to be -1 by default, which (in all scripts) will pick a random one, as in eval_approach. TODO: Put time_estimate out of the pipeline, in a different library (it already existed).

class driftai.project.Project(name=None, path=None, creation_date=None)[source]¶

Bases: object

Parameters:	name (str) – Project name. Default: untitled_driftai_project path (str) – Project path creation_date (datetime) – Creation date. Should not be set manually
Raises:	`OptAppProjectNameExistsException` – In case project name already exists

dir_components = {'approaches': 'dir', 'driftai.db': 'file', 'project_files': 'dir'}¶

dir_exceptions = {'default': <class 'driftai.exceptions.OptAppProjectElementNotExistsException'>, 'dir': <class 'driftai.exceptions.OptAppProjectDirNotExistsException'>, 'file': <class 'driftai.exceptions.OptAppProjectFileNotExistsException'>}¶

exists()[source]¶

Check if project exists

Returns:	True if project already exists
Return type:	bool

get_info()[source]¶

Get project info

Returns:	Dictionary containing the project info: { "path": <project_location>, "creation_date": <project's creation date>, "name": <project name> }
Return type:	dict

get_last_subdataset()[source]¶

Get last subdataset

Returns:
Return type:	driftai.data.SubDataset

get_subdatasets()[source]¶

Get all subdatasets

Returns:	All subdatasets related to current project
Return type:	list(Subdataset)

id¶

is_running()[source]¶

classmethod load()[source]¶

Creates a project from a tinydb file

Parameters:	path (str) – Project location
Raises:	`OptAppProjectWrongInfoFileStructureException` – In case project structure is not valid
Returns:	Returns the loaded project
Return type:	Project

save()[source]¶

driftai.utils module¶

driftai.utils.check_folder_structure(path_to_dir, content_dict, exception_dict)[source]¶

Check if the directory structure is the expected. In case folder structure is invalid raises the exception specified inside exception_dict

Parameters:	path_to_dir (dict) – Directory path path_to_dir – Dictionary which its keys are the expected contents, and its values are the contents’ file type (file \| dir) exception_dict (dict) – Dictionary containing the exceptions to be thrown in case a file or a directory is missing

driftai.utils.check_uri(uri)[source]¶

Checks if a string is an uri

Parameters:	uri (str) – String to be tested
Returns:	True -> If uri is an uri False -> If uri is not an uri
Return type:	bool

driftai.utils.compile_path_pattern(p, basepattern='', all_path_matcher_name='')[source]¶

Convert a pattern expression for the data path to regular expression

Parameters:

p (str) – Path pattern expression. Example: ‘{testset}/{class}_{}.png|jpg’
basepattern (str) – Base pattern. This is a fixed part of the pattern (like the root path)

all_path_matcher_name (str) –

Name to name the group matching to all path pattern expression: Example

>>> regex = compile_path_pattern('{testset}/{class}/{}.png|jpg',
... basepattern="/home/users/, all_path_matcher_name='file')
>>> g = re.match(regex, '/home/users/train/0/0.png')
>>> print(g)
{
    ...
    'file': 'train/0/0.png'
    ...
}

driftai.utils.filepath_to_uri(filepath)[source]¶

Converts a system path to an uri file

Parameters:	filepath (str) – Path to be converted

Example

filepath_to_uri(/home/my_file.txt) -> file:///home/myfile.txt

driftai.utils.get_current_project_db(path)[source]¶

Get project database

Parameters:	path (str) – Any path inside driftai project
Returns:
Return type:	driftai.db.Database

driftai.utils.get_file_extension(filename)[source]¶

Returns the file extension without the dot

Parameters:	filename (str) – Path to file
Returns:	File extension without the ‘.’
Return type:	str

driftai.utils.import_from(module, name)[source]¶

Imports an attribute from a module.

Parameters:	module (str) – Module namespace. For example: sklearn.model_selection name (str) – Name of attribute. For example train_test_split
Returns:	Attribute if found else None
Return type:	any

driftai.utils.maybe_make_dir(path)[source]¶

Creates a directory if it does not exist

Parameters:	path (pathlib.Path) – Directory path

driftai.utils.print_progress_bar(iteration, total, prefix='', suffix='')[source]¶

Call in a loop to create terminal progress bar

Parameters:	iteration (int) – Current iteration total (int) – Total iterations prefix (str) – Prefix string suffix (str) – Suffix string

driftai.utils.str_to_date(in_str)[source]¶

Converts a string date to datetime. If the parameter is a datetime returns it without any transformation else if the parameter is a string casts it to a datetime with the following format: %Y-%m-%d %H:%M:%S.%f :param in_str: String to be casted as datetime :type in_str: str or datetime

Returns:	Casted string
Return type:	datetime

driftai.utils.to_camel_case(str_name, separator_values=['-', '_'])[source]¶

Converts an string to camel case naming convention

Parameters:	str_name (str) – String to be converted separator_values (list of str, optional) – String separators
Returns:	String in camel case format
Return type:	str

Example

to_camel_case(‘random_forest’, spearator_values=[‘_’]) -> RandomForest

driftai.utils.uri_to_filepath(uri)[source]¶

Transforms a file uri to path system

Parameters:	uri (str) – String to be transformed
Returns:	Converted string
Return type:	str

Example

uri_to_filepath(“file:///home/myfile.txt”) -> /home/myfile.txt

driftai package¶

Subpackages¶

Submodules¶

driftai.approach module¶

driftai.exceptions module¶

driftai.project module¶

driftai.utils module¶

Table of Contents

Previous topic

Next topic

This Page