scicloj.metamorph.ml.evaluation-handler
Categories
Other vars: default-result-dissoc-in-fn default-result-dissoc-in-seq example-nippy-handler get-source-information metrics-and-model-keep-fn metrics-and-options-keep-fn metrics-keep-fn qualify-keywords qualify-pipelines result-dissoc-in-seq--all result-dissoc-in-seq--all-fn result-dissoc-in-seq--ctxs result-dissoc-in-seq-ctx-fn
default-result-dissoc-in-fn
(default-result-dissoc-in-fn result)default evaluation-handler-fn
default-result-dissoc-in-seq
Default sequence of key paths to remove from evaluation results.
Removes large data objects (datasets, model internals) while preserving metrics, options, and essential model metadata. Removes:
- Dataset objects from fit/train/test contexts
- Target and feature datasets from model
- Smile-specific model internals (serialized bytes, dataframes)
Used by default-result-dissoc-in-fn to clean results for storage/analysis.
example-nippy-handler
(example-nippy-handler files output-dir result-reduce-fn)Creates an evaluation result handler that serializes results to Nippy format.
files- Atom to track generated file pathsoutput-dir- Directory path for output filesresult-reduce-fn- Function to apply to results after serialization
Returns a handler function that removes non-freezable data (:pipe-fn, :metric-fn), serializes the result to a UUID-named .nippy file, and applies result-reduce-fn.
See also: scicloj.metamorph.ml/evaluate-pipelines
get-source-information
(get-source-information qualified-pipe-decl pipe-ns pipeline-source-file)Creates metadata about a pipeline including function sources and classpath.
qualified-pipe-decl- Pipeline declaration with fully-qualified keywordspipe-ns- Namespace symbol for the pipelinepipeline-source-file- Path to the source file containing the pipeline
Returns a map with :fn-sources (source code of all pipeline functions) and :classpath (JVM classpath at evaluation time).
metrics-and-model-keep-fn
(metrics-and-model-keep-fn result)evaluation-handler-fn which keeps only train-metric, test-metric and the fitted model map, which contains as well the model object as byte array (amon other things)
metrics-and-options-keep-fn
(metrics-and-options-keep-fn result)evaluation-handler-fn which keeps only train-metric, test-metric and and the options
metrics-keep-fn
(metrics-keep-fn result)evaluation-handler-fn which keeps only train-metric, test-metric
qualify-keywords
(qualify-keywords pipe-decl pipe-ns)Converts unqualified keywords in a pipeline declaration to fully-qualified form.
pipe-decl- Pipeline declaration (nested data structure)pipe-ns- Namespace symbol for keyword resolution
Returns the pipeline declaration with all resolvable keywords converted to namespace-qualified keywords (e.g., :fn-name becomes :my.ns/fn-name).
Used to make pipeline declarations portable across namespaces.
qualify-pipelines
(qualify-pipelines pipe-decls pipe-ns)Qualifies all keywords in a sequence of pipeline declarations.
pipe-decls- Sequence of pipeline declarationspipe-ns- Namespace symbol for keyword resolution
Returns a vector of qualified pipeline declarations. Applies qualify-keywords to each pipeline in the sequence.
See also: scicloj.metamorph.ml.evaluation-handler/qualify-keywords
result-dissoc-in-seq--all
Maximum cleanup: removes almost everything except core metrics and model type.
Removes:
- contexts
- timing data
- probability distributions
- pipeline declarations
- functions
- source information
Keeps only the essential metric values and model type.
Most aggressive cleanup option. Used by result-dissoc-in-seq--all-fn.
result-dissoc-in-seq--all-fn
(result-dissoc-in-seq--all-fn result)evaluation-handler-fn which removes all :ctx
result-dissoc-in-seq--ctxs
Sequence of paths to remove all context objects from evaluation results.
Removes fit, train, and test contexts entirely, keeping only metrics and metadata. More aggressive cleanup than default-result-dissoc-in-seq.
Used by result-dissoc-in-seq-ctx-fn.
result-dissoc-in-seq-ctx-fn
(result-dissoc-in-seq-ctx-fn result)evaluation-handler-fn which removes all :ctx