Skip to content

Commit

Permalink
update pipeline
Browse files Browse the repository at this point in the history
  • Loading branch information
xingzhongyu committed Jun 8, 2024
1 parent 939a268 commit 8255f8f
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions dance/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -1084,11 +1084,13 @@ def run_step3(root_path, evaluate_pipeline, step2_pipeline_planer: PipelinePlane

pipeline_top_k = default(step2_pipeline_planer.config.pipeline_tuning_top_k, DEFAULT_PIPELINE_TUNING_TOP_K)
step3_k = default(step2_pipeline_planer.config.parameter_tuning_freq_n, DEFAULT_PARAMETER_TUNING_FREQ_N)
# Skip some of the already run step3 because in pandas, when you sort columns with exactly the same values, the results are not random.
# Instead, pandas preserves the order of the original data. So we can skip it without causing any impact.
step3_start_k=default(step2_pipeline_planer.config.step3_start_k, 0)
#Some sweep_ids of step3 that have already been run
step3_sweep_ids=step2_pipeline_planer.config.step3_sweep_ids
step3_sweep_ids=[None] * len(pipeline_top_k) if step3_sweep_ids is None else (step3_sweep_ids + [None] * (pipeline_top_k - len(step3_sweep_ids)))
# Skip some of the already run step3 because in pandas, when you sort columns with exactly the same values, the results are not random.
# Instead, pandas preserves the order of the original data. So we can skip it without causing any impact.

for i in range(pipeline_top_k):
if i < step3_start_k:
continue
Expand Down

0 comments on commit 8255f8f

Please sign in to comment.