PyAutoNervesBoard

Signal. Connect. Understand.

PyAutoFit · autofit/config/

← all sources · library · 17 files · 562 lines · GitHub

(top level) (4)

general.yaml

36 lines · 36 keys · settings · 29 used · 2 section-read · 5 unused · GitHub

updateshpcinversionoutputparallelprofilingtest
source (36 lines)
1updates:2  iterations_per_quick_update: 1e99 # Non-linear search iterations between every quick update, which just displays the maximum likelihood model fit.3  iterations_per_full_update: 1e99  # Non-linear search iterations between every full update, which outputs all visuals and result fits (e.g. model.result, search.summary), this exits the search and can be slow.4  quick_update_background: false    # If True, quick-update visualization runs on a background thread so sampling is not blocked.5  live_visual_update: false         # If True, quick-update visuals are pushed to a live surface (a Jupyter cell when in a kernel, a matplotlib viewer subprocess otherwise) in addition to being written to disk.6hpc:7  hpc_mode: false                   # If True, use HPC mode, which disables GUI visualization, logging to screen and other settings which are not suited to running on a super computer.8  iterations_per_quick_update: 1e99 #  Non-linear search iterations between every quick update, which just displays the maximum likelihood model fit.9  iterations_per_full_update: 1e99  # Non-linear search iterations between every full update, which outputs all visuals and result fits (e.g. model.result, search.summary), this exits the search and can be slow.10inversion:11  check_reconstruction: true        # If True, the inversion's reconstruction is checked to ensure the solution of a meshs's mapper is not an invalid solution where the values are all the same.12  reconstruction_vmax_factor: 0.5   # Plots of an Inversion's reconstruction use the reconstructed data's bright value multiplied by this factor.13output:14  force_pickle_overwrite: false     # If True pickle files output by a search (e.g. samples.pickle) are recreated when a new model-fit is performed.15  force_visualize_overwrite: false  # If True, visualization images output by a search (e.g. subplots of the fit) are recreated when a new model-fit is performed.16  info_whitespace_length: 80        # Length of whitespace between the parameter names and values in the model.info / result.info17  log_level: INFO                   # The level of information output by logging. unused18  log_to_file: false                # If True, outputs the non-linear search log to a file (and not printed to screen). unused19  log_file: output.log              # The name of the file the logged output is written to (in the non-linear search output folder) unused20  model_results_decimal_places: 3   # Number of decimal places estimated parameter values / errors are output in model.results.21  remove_files: false               # If True, all output files of a non-linear search (e.g. samples, visualization, etc.) are deleted once the model-fit has completed, such that only the .zip file remains.22  search_internal : false    # If True, the search internal folder which contains a saved state of the non-linear search is delete, saving on hard-disk space. unused23  samples_to_csv: true              # If True, non-linear search samples are written to a .csv file.24  unconverged_sample_size : 100     # If outputting results of an unconverged search, the number of samples used to estimate the median PDF values and errors.25  visualize_ep_factor_searches: false  # Expectation propagation only. If True, every EP factor search draws its own per-search visuals (e.g. corner plots, model-fit images) and redraws its before-fit visuals. If False, only the first search of each factor draws before-fit visuals and no factor search draws per-search visuals (samples and summaries are still written; the EP optimiser's own graph.png / ep_history output is unaffected).26parallel:27  warn_environment_variables: true  # If True, a warning is displayed when the search's number of CPU > 1 and enviromment variables related to threading are also > 1.28profiling:29  parallel_profile: false           # If True, the parallelization of the fit is profiled outputting a cPython graph.30  repeats: 1                        # The number of repeat function calls used to measure run-times when profiling. unused31test:32  check_likelihood_function: true   # if True, when a search is resumed the likelihood of a previous sample is recalculated to ensure it is consistent with the previous run.33  exception_override: false34  lh_timeout_seconds:               # If a float is input, the log_likelihood_function call is timed out after this many seconds, to diagnose infinite loops. Default is None, meaning no timeout.35  log_likelihood_ceiling:           # If a float is input, log likelihoods whose magnitude exceeds it are treated as numerical garbage and replaced by the resample figure of merit. Blank (null) or inf disables the guard, which is the default: a log likelihood scales with the noise-map units, so a fixed magnitude can reject a legitimate fit on a badly-scaled dataset. Opt in where the units are known (e.g. autolens_profiling sets 1.0e+20).36  parallel_profile: false

logging.yaml

19 lines · 15 keys · settings · 1 used · 14 section-read · 0 unused · GitHub

versiondisable_existing_loggershandlersrootformatterstotal_files_open
source (19 lines)
1version: 12disable_existing_loggers: false34handlers:5  console:6    class: logging.StreamHandler7    level: INFO8    stream: ext://sys.stdout9    formatter: formatter1011root:12  level: INFO13  handlers: [ console ]1415formatters:16  formatter:17    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'1819total_files_open : false # Whether to output info on the total number of open files in the system, used for debugging parallelism issues.

notation.yaml

41 lines · 23 keys · settings · 5 used · 18 section-read · 0 unused · GitHub

labellabel_format
source (41 lines)
1# The notation configs define the labels of every model parameter which are used when2# visualizing results (for example labeling the axis of the PDF triangle plots output by a non-linear search).345# label: The label given to the each parameter, for plots like PDF corner plots.67# For example, if `centre=x`, the plot axis will be labeled 'x'.8910# superscript: the superscript used on certain plots that show the results of different model-components.1112# For example, if `Gaussian=g`, plots where the parameters of the Gaussian model-component have superscript `g`.1314label:15  label:16    centre: x17    normalization: norm18    parameter0: a19    parameter1: b20    parameter2: c21    rate: \lambda22    sigma: \sigma23  superscript:24    Exponential: e25    Gaussian: g26    ModelComponent0: M027    ModelComponent1: M12829# label_format: The format certain parameters are output as in output files like the `model.results` file.3031# For example, if `centre={:.2d}`, the format of the centre parameter in results files will use this Python format.3233label_format:34  format:35    centre: '{:.2f}'36    normalization: '{:.2f}'37    parameter0: '{:.2f}'38    parameter1: '{:.2f}'39    parameter2: '{:.2f}'40    rate: '{:.2f}'41    sigma: '{:.2f}'

output.yaml

110 lines · 17 keys · settings · 12 used · 5 section-read · 0 unused · GitHub

defaultsamplessamples_weight_thresholdsearch_internalstart_pointlatent_during_fitlatent_after_fitlatent_draw_via_pdflatent_draw_via_pdf_sizelatent_csvlatent_resultscovariancedatanoise_mapsearch_logmodel_graphmodel_figure
source (110 lines)
1# Determines whether files saved by the search are output to the hard-disk. This is true both when saving to the2# directory structure and when saving to database.34default: true # If true then files which are not explicitly listed here are output anyway. If false then they are not.56### Samples ###78# The `samples.csv`file contains every sampled value of every free parameter with its log likelihood and weight.910# This file is often large, therefore disabling it can significantly reduce hard-disk space use.1112# `samples.csv` is used to perform marginalization, infer model parameter errors and do other analysis of the search13# chains. Even if output of `samples.csv` is disabled, these tasks are still performed by the fit and output to14# the `samples_summary.json` file. However, without a `samples.csv` file these types of tasks cannot be performed15# after the fit is complete, for example via the database.1617samples: true1819# The `samples.csv` file contains every accepted sampled value of every free parameter with its log likelihood and20# weight. For certain searches, the majority of samples have a very low weight and have no numerical impact on the21# results of the model-fit. However, these samples are still output to the `samples.csv` file, taking up hard-disk22# space and slowing down analysis of the samples (e.g. via the database).2324# The `samples_weight_threshold` below specifies the threshold value of the weight such that samples with a weight25# below this value are not output to the `samples.csv` file. This can be used to reduce the size of the `samples.csv`26# file and speed up analysis of the samples.2728# For many searches (e.g. MCMC) all samples have an equal weight of 1.0, and this threshold therefore has no impact.29# For these searches, there is no simple way to save hard-disk space. This input is more suited to nested sampling,30# where the majority of samples have a very low weight..3132# Set value to empty (e.g. delete 1.0e-10 below) to disable this feature.3334samples_weight_threshold: 1.0e-103536### Search Internal ###3738# The search internal folder which contains a saved state of the non-linear search in its internal reprsenetation,39# as a .pickle or .dill file.4041# For example, for the nested sampling dynesty, this .dill file is the `DynestySampler` object which is used to42# perform sampling, and it therefore contains all internal dynesty representations of the results, samples, weights, etc.4344# If the entry below is false, the folder is still output during the model-fit, as it is required to resume the fit45# from where it left off. Therefore, settings `false` below does not impact model-fitting checkpointing and resumption.46# Instead, the search internal folder is deleted once the fit is completed.4748# The search internal folder file is often large, therefore deleting it after a fit is complete can significantly49# reduce hard-disk space use.5051# The search internal representation that can be loaded from the .dill file has many additional quantities specific to52# the non-linear search that the standardized autofit forms do not. For example, for emcee, it contains information on53# every walker. This information is required to do certain analyes and make certain plots, therefore deleting the54# folder means this information is list.5556search_internal: false5758### Start Point ###5960# If an Initalizer is used to provide a start point for the non-linear search, visualization of that start point can be61# output to hard-disk to show the user the initial model-fit that is used to start the search. This visualization is62# the visualizer wrapped in the Analysis class, and therefore should show things like the quality of the fit63# to the data and the residuals at the start point.6465start_point: true6667### Latent Variables ###6869# A latent variable is not a model parameter but can be derived from the model. Its value and errors may be of interest70# and aid in the interpretation of a model-fit.7172# For example, for the simple 1D Gaussian example, it could be the full-width half maximum (FWHM) of the Gaussian. This73# is not included in the model but can be easily derived from the Gaussian's sigma value.7475# By overwriting an Analysis class's `compute_latent_variables` method we can manually specify latent variables that76# are calculated and output to a `latent.csv` file, which mirrors the `samples.csv` file. The `latent.csv` file has77# the same weight resampling performed on the `samples.csv` file, controlled via the `samples_weight_threshold` above.7879# There may also be a `latent.results` and `latent_summary.json` files output, which the inputs below control whether80# they are output and how often.8182# Outputting latent variables manually after a fit is complete is simple, just call83# the `analysis.compute_latent_variables()` function.8485# For many use cases, the best set up may be to disable autofit latent variable output during the fit and perform it86# manually after completing a successful model-fit. This will save computational run time by not computing latent87# variables during a any model-fit which is unsuccessful.8889latent_during_fit: false # Whether to output the `latent.csv`, `latent.results` and `latent_summary.json` files during the fit when it performs on-the-fly output.90latent_after_fit: true # If `latent_during_fit` is False, whether to output the `latent.csv`, `latent.results` and `latent_summary.json` files after the fit is complete.91latent_draw_via_pdf : true # Whether to draw latent variable values via the PDF of every sample, which uses fewer samples to estimate latent variable errors. If False, latent variable values are drawn from every sample.92latent_draw_via_pdf_size : 100 # The number of samples drawn to estimate latent variable errors if `latent_draw_via_pdf` is True.93latent_csv: true # Whether to ouptut the `latent.csv` file.94latent_results: true # Whether to output the `latent.results` file.9596# Other Files:9798covariance: true # `covariance.csv`: The [free parameters x free parameters] covariance matrix.99data: true # `data.json`: The value of every data point in the data.100noise_map: true # `noise_map.json`: The value of every RMS noise map value.101102search_log: true # `search.log`: logging produced whilst running the fit method103104model_graph: false # `model.graph`: graphical-model factor graph summary. Only meaningful for graphical models (autofit.graphical); empty for ordinary PriorModel/Collection fits, so disabled by default.105106# `model.png`: the model figure drawn beside `model.info` (autofit.model_figure / af.ModelPlotter).107# Opt-in until the lens acceptance renders pass (model-figures epic phase 3). Unlike other keys this108# one does NOT fall back to `default` when absent -- an absent key means off.109110model_figure: false

non_linear (1)

non_linear/GridSearch.yaml

5 lines · 3 keys · settings · 0 used · 0 section-read · 3 unused · GitHub

parallel
source (5 lines)
1# The settings of a parallelized grid search of non-linear searches.23parallel: unused4  number_of_cores: 3 # The number of cores the search is parallelized over by default, using Python multiprocessing. unused5  step_size: 0.1     # The default step size of each grid search parameter, in terms of unit values of the priors. unused

priors (9)

priors/Exponential.yaml

30 lines · 30 keys · prior file · GitHub

centrenormalizationrate
priors (6 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
centrelimits
centrewidth_modifierAbsolutevalue 20.0
normalizationlimits
normalizationwidth_modifierRelativevalue 0.5
ratelimits
ratewidth_modifierRelativevalue 0.5

priors/Gaussian.yaml

30 lines · 30 keys · prior file · GitHub

centrenormalizationsigma
priors (6 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
centrelimits
centrewidth_modifierAbsolutevalue 20.0
normalizationlimits
normalizationwidth_modifierRelativevalue 0.5
sigmalimits
sigmawidth_modifierRelativevalue 0.5

priors/Gaussian2D.yaml

40 lines · 40 keys · prior file · GitHub

centre_0centre_1normalizationsigma
priors (8 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
centre_0limits
centre_0width_modifierAbsolutevalue 20.0
centre_1limits
centre_1width_modifierAbsolutevalue 20.0
normalizationlimits
normalizationwidth_modifierRelativevalue 0.5
sigmalimits
sigmawidth_modifierRelativevalue 0.5

priors/GaussianKurtosis.yaml

40 lines · 40 keys · prior file · GitHub

centrekurtosisnormalizationsigma
priors (8 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
centrelimits
centrewidth_modifierAbsolutevalue 20.0
kurtosislimits
kurtosiswidth_modifierAbsolutevalue 20.0
normalizationlimits
normalizationwidth_modifierRelativevalue 0.5
sigmalimits
sigmawidth_modifierRelativevalue 0.5

priors/MultiLevelGaussians.yaml

10 lines · 10 keys · prior file · GitHub

higher_level_centre
priors (2 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
higher_level_centrelimits
higher_level_centrewidth_modifierAbsolutevalue 20.0

priors/model.yaml

62 lines · 62 keys · prior file · GitHub

ExponentialGaussian
priors (6 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
ExponentialcentreUniformlower 0.0upper 100.0Absolute 20.0[-inf, inf]
ExponentialnormalizationLogUniformlower 1e-06upper 1000000.0Relative 0.5[0.0, inf]
ExponentialrateUniformlower 0.0upper 10.0Relative 0.5[0.0, inf]
GaussiancentreUniformlower 0.0upper 100.0Absolute 20.0[-inf, inf]
GaussiannormalizationLogUniformlower 1e-06upper 1000000.0Relative 0.5[0.0, inf]
GaussiansigmaUniformlower 0.0upper 25.0Relative 0.5[0.0, inf]

priors/prior.yaml

7 lines · 7 keys · prior file · GitHub

gaussian.GaussianPrior
priors (2 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
gaussian.GaussianPriorlower_limitConstantvalue -inf
gaussian.GaussianPriorupper_limitConstantvalue inf

priors/profiles.yaml

62 lines · 62 keys · prior file · GitHub

ExponentialGaussian
priors (6 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
ExponentialcentreUniformlower 0.0upper 100.0Absolute 20.0[-inf, inf]
ExponentialnormalizationLogUniformlower 1e-06upper 1000000.0Relative 0.5[0.0, inf]
ExponentialrateUniformlower 0.0upper 10.0Relative 0.5[0.0, inf]
GaussiancentreUniformlower 0.0upper 100.0Absolute 20.0[-inf, inf]
GaussiannormalizationLogUniformlower 1e-06upper 1000000.0Relative 0.5[0.0, inf]
GaussiansigmaUniformlower 0.0upper 25.0Relative 0.5[0.0, inf]

priors/template.yaml

53 lines · 53 keys · prior file · GitHub

ModelComponent0ModelComponent1
priors (6 params)
ClassParamTypeCentre / lowerWidth / upperWidth modifierLimits
ModelComponent0parameter0Uniformlower 0.0upper 1.0Absolute 20.0
ModelComponent0parameter1LogUniformlower 1e-06upper 1000000.0Relative 0.5
ModelComponent0parameter2Uniformlower 0.0upper 25.0Relative 0.5
ModelComponent1parameter0Uniformlower 0.0upper 1.0Absolute 20.0[-inf, inf]
ModelComponent1parameter1LogUniformlower 1e-06upper 1000000.0Relative 0.5[0.0, inf]
ModelComponent1parameter2Uniformlower 0.0upper 1.0Relative 0.5[0.0, inf]

visualize (3)

visualize/general.yaml

2 lines · 2 keys · settings · 1 used · 1 section-read · 0 unused · GitHub

general
source (2 lines)
1general:2  backend: default         # The matploblib backend used for visualization. `default` uses the system default, can specifiy specific backend (e.g. TKAgg, Qt5Agg, WXAgg).

visualize/plots_search.yaml

8 lines · 8 keys · settings · 3 used · 5 section-read · 0 unused · GitHub

nestmcmcmle
source (8 lines)
1nest:2  corner_anesthetic: true   # Output corner figure (using anestetic) during a non-linear search fit?3mcmc:4  corner_cornerpy: true     # Output corner figure (using corner.py) during a non-linear search fit?5mle:6  subplot_parameters: true   # Output a subplot of the best-fit parameters of the model?7  log_likelihood_vs_iteration: true  # Output a plot of the log likelihood versus iteration number?8  figure_of_merit_vs_iteration: true  # Output the global-best figure-of-merit trace (auto-convergence gradient searches)?

visualize/plots_settings.yaml

7 lines · 7 keys · settings · 7 used · 0 section-read · 0 unused · GitHub

corner_anestheticcorner_cornerpy
source (7 lines)
1corner_anesthetic:2  figsize_per_parammeter: 4   # The figsize of the matplotlib figure is given as the number of parameters times this value, for exmaple with 3 free parameters and a value 4 the figsize will be 3*4 = 12.3  fontsize: 20               # The size of the font of the tick values and parameter labels on the x and y axis of the corner plot.4  facecolor: white           # The facecolor of the corner plot.5  alpha: 0.9                 # The alpha value of the corner plot.6corner_cornerpy:7  fontsize: 14               # The size of the font of the tick values and parameter labels on the x and y axis of the corner plot.