chariot.evaluation package

Submodules

chariot.evaluation.model_comparison module

class chariot.evaluation.model_comparison.GPSBox(*, min_lat: float | int, min_lon: float | int, max_lat: float | int, max_lon: float | int)[source]

Bases: GPSBox

GPS box coordinates.

Attributes

min_latfloat | int: Minimum latitude.
min_lonfloat | int: Minimum longitude.
max_latfloat | int: Maximum latitude.
max_lonfloat | int: Maximum longitude.

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class chariot.evaluation.model_comparison.MetadataFilter(*, key: str, operator: str, value: str)[source]

Bases: MetadataFilter

ISO-8601 format time boundaries.

Attributes

keystr: Metadata key to perform filtering upon.
operatorstr: Operator symbol.
valuestr: The value to compare by.

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class chariot.evaluation.model_comparison.TimeRange(*, start: str | None = None, end: str | None = None)[source]

Bases: TimeRange

ISO-8601 format time boundaries.

Attributes

startstr, optional: Optional start time in ISO-8601 format.
endstr, optional: Optional end time in ISO-8601 format.

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

chariot.evaluation.model_comparison.evaluate(job_ids: list[str], filter_by_location: GPSBox | None = None, filter_by_time_range: TimeRange | None = None, filter_by_metadata: list[MetadataFilter] | None = None, aggregate_over_labels: list[str] | None = None) → dict[source]

Run evaluation with option to filter data.

Parameters

job_idslist[str]: A list of bulk inference job ids to perform evaluation over.
filter_by_locationfilter_by_location, optional: Optionally filter datums by location.
filter_by_time_rangeTimeRange, optional: Optionally filter datums by time of creation.
filter_by_metadatalist[MetadataFilter], optional: Optionally filter datums by metadata.
aggregate_over_labelslist[str], optional: Optionally create metrics over labels subsets.

Returns

dict: The evaluation response.

chariot.evaluation.model_comparison.get_metadata(job_ids: list[str]) → list[dict[str, Any]][source]

Get dataset metadata.

Parameters

job_idslist[str]: A list of bulk inference job ids.

chariot.evaluation.model_comparison.get_metadata_keys(job_ids: list[str]) → dict[str, Any][source]

Get dataset metadata keys.

Parameters

job_idslist[str]: A list of bulk inference job ids.

Module contents

class chariot.evaluation.GPSBox(*, min_lat: float | int, min_lon: float | int, max_lat: float | int, max_lon: float | int)[source]

Bases: GPSBox

GPS box coordinates.

Attributes

min_latfloat | int: Minimum latitude.
min_lonfloat | int: Minimum longitude.
max_latfloat | int: Maximum latitude.
max_lonfloat | int: Maximum longitude.

max_lat: StrictFloat | StrictInt

max_lon: StrictFloat | StrictInt

min_lat: StrictFloat | StrictInt

min_lon: StrictFloat | StrictInt

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class chariot.evaluation.MetadataFilter(*, key: str, operator: str, value: str)[source]

Bases: MetadataFilter

ISO-8601 format time boundaries.

Attributes

keystr: Metadata key to perform filtering upon.
operatorstr: Operator symbol.
valuestr: The value to compare by.

key: StrictStr

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

operator: StrictStr

value: StrictStr

class chariot.evaluation.TimeRange(*, start: str | None = None, end: str | None = None)[source]

Bases: TimeRange

ISO-8601 format time boundaries.

Attributes

startstr, optional: Optional start time in ISO-8601 format.
endstr, optional: Optional end time in ISO-8601 format.

end: StrictStr | None

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

start: StrictStr | None

chariot.evaluation.evaluate(job_ids: list[str], filter_by_location: GPSBox | None = None, filter_by_time_range: TimeRange | None = None, filter_by_metadata: list[MetadataFilter] | None = None, aggregate_over_labels: list[str] | None = None) → dict[source]

Run evaluation with option to filter data.

Parameters

job_idslist[str]: A list of bulk inference job ids to perform evaluation over.
filter_by_locationfilter_by_location, optional: Optionally filter datums by location.
filter_by_time_rangeTimeRange, optional: Optionally filter datums by time of creation.
filter_by_metadatalist[MetadataFilter], optional: Optionally filter datums by metadata.
aggregate_over_labelslist[str], optional: Optionally create metrics over labels subsets.

Returns

dict: The evaluation response.