penguin.genetic module

class penguin.genetic.ConfigChromosome(parent=None, new_gene=None)[source]

Bases: object

Parameters:

parent (ConfigChromosome)

genes: frozenset[MitigationPatch]
get_mitigation(failure_name)[source]
hash: int
mitigations_to_str()[source]
to_dict()[source]
class penguin.genetic.ConfigPopulation(proj_dir, base_config, run_base, logger, pop_size, timeout, max_runs, nelites=1, verbose=False)[source]

Bases: ConfigSearch

config_in_pop(config)[source]
Parameters:

config (ConfigChromosome)

create_work_queue(nworkers, chromosomes)[source]
Parameters:
crossover(p1_prob=0.5)[source]

Perform crossover on the parents to create new children, choose from p1 with probability p1_prob

extend_genome(parent, new_gene)[source]
Parameters:
get_fitness(config)[source]
Parameters:

config (ConfigChromosome)

get_full_config(config)[source]

Given a configuration, return the full configuration (base_config+config)

Parameters:

config (ConfigChromosome)

get_patched_config(config)[source]
Parameters:

config (ConfigChromosome)

join_workers(nworkers=1)[source]
mutation(nmuts=1)[source]

Perform mutation on the population, we are going to be a bit more aggressive with mutation than traditional GAs

Our mutation here is essentially “try a different mitigation”. Elites will get modified here

print_chromosomes(chromosomes)[source]
record_fitness(config, fitness)[source]
Parameters:

config (ConfigChromosome)

run_config(config, run_index)[source]

Careful! This function is run in parallel and should not modify any shared state.

This is very dirty - we return the failure type from graph.py, not our own Failure class

We also tack on the base config, so watch out for that.

Parameters:
Return type:

Tuple[List[Failure], float]

run_configs(nthreads, chromosomes)[source]
Parameters:
run_worker(id, results)[source]
Parameters:

  • id (int)

  • results (dict)

selection(nparents)[source]

A ranked based selection. We’ll choose nparents to move on to the next generation

By default, hold on to the highest fitness config (nelites=1)

class penguin.genetic.GenePool[source]

Bases: object

The gene pool tracks the set of possible mitigations for a given failure

failure_to_genename()[source]
Parameters:

failure (Failure)

get_mitigations(failure)[source]
get_names()[source]
update(new_gene)[source]
class penguin.genetic.MitigationAlleleSet(failure_name, mitigations)[source]

Bases: object

This class contains a set of mitigations we know about for a given failure In the biology analogy, each mitigation would be an allele

Parameters:

  • failure_name (str)

  • mitigations (frozenset)

failure_name: str
mitigations: frozenset
class penguin.genetic.MitigationPatch(failure_name, patch)[source]

Bases: object

This class represents a single mitigation and the patch to apply

Parameters:

  • failure_name (str)

  • patch (frozenset)

failure_name: str
patch: frozenset
penguin.genetic.create_init_gene(base_config, proj_dir, patch_dir)[source]

Based on add_init_options_to_graph in manager.py

A config needs to have an [‘env’][‘igloo_init’] in order to do anything useful. We might have a single option already set or we might have multiple options stored proj_dir/static/InitFinder.yaml (based on static analysis).

If we have no value set and no potential values, we raise an error.

Otherwise we’ll create a fake failure for “init” and a mitigation node to add each of the init options. Then we’ll create configs with each init. This means we’ll start our search with multiple configuration options to explore.

If an igloo_init is set in the initial config, we’ll assume that’s right and leave it alone.

penguin.genetic.diff_configs(new, old)[source]

Main entrypoint. Given an initial config and directory run our genetic algorithm search.

penguin.genetic.main()[source]