test_padis_gcp_spot_training.py

Source: tests/experiments/test_padis_gcp_spot_training.py

Test padis gcp spot training behaviour.

LION_ROOT

GCP_RUNNER

GCP_MANUAL_RECONSTRUCTION

GCP_STARTUP

GCP_METADATA_STARTUP

GCP_SHUTDOWN

DEFAULT_GCP_RUN_NAME

test_gcp_spot_runner_is_executable()

Verify that gcp spot runner is executable.

test_gcp_manual_reconstruction_dry_run_uses_bucket_mount_paths(tmp_path)

Verify that gcp manual reconstruction dry run uses bucket mount paths.

test_gcp_spot_runner_dry_run_builds_expected_training_commands(tmp_path)

Verify that gcp spot runner dry run builds expected training commands.

test_gcp_spot_runner_default_order_runs_p96_immediately_after_pnp(tmp_path)

Verify that gcp spot runner default order runs p96 immediately after pnp.

test_gcp_spot_runner_adds_validation_heavy_continuation_phase(tmp_path)

Verify that gcp spot runner adds validation heavy continuation phase.

test_gcp_spot_runner_validation_phase_resumes_base_done_tasks(tmp_path)

Verify that gcp spot runner validation phase resumes base done tasks.

test_gcp_spot_runner_validation_phase_uses_separate_runtime_ledger(tmp_path)

Verify that gcp spot runner validation phase uses separate runtime ledger.

test_gcp_spot_runner_default_run_name_and_wandb_prefix(tmp_path)

Verify that gcp spot runner default run name and wandb prefix.

test_gcp_spot_runner_defaults_to_one_gpu(tmp_path)

Verify that gcp spot runner defaults to one gpu.

test_gcp_spot_runner_uses_128_batch_for_p96_by_default(tmp_path)

Verify that gcp spot runner uses 128 batch for p96 by default.

test_gcp_spot_runner_subtracts_previous_runtime_from_wall_budget(tmp_path)

Verify that gcp spot runner subtracts previous runtime from wall budget.

test_gcp_spot_runner_uses_finalise_window_when_budget_is_exhausted(tmp_path)

Verify that gcp spot runner uses finalise window when budget is exhausted.

test_gcp_shutdown_hook_refreshes_active_runtime_ledger(tmp_path)

Verify that gcp shutdown hook refreshes active runtime ledger.

test_gcp_startup_hook_dry_run_prepares_env_and_runner_command(tmp_path)

Verify that gcp startup hook dry run prepares env and runner command.

test_gcp_metadata_startup_bootstrap_delegates_after_data_mount(tmp_path)

Verify that gcp metadata startup bootstrap delegates after data mount.

test_gcp_spot_runner_records_final_and_full_checkpoints(tmp_path)

Verify that gcp spot runner records final and full checkpoints.

test_gcp_spot_runner_runs_resumable_reconstruction_phase(tmp_path)

Verify that gcp spot runner runs resumable reconstruction phase.

test_gcp_spot_runner_allows_multiple_reconstruction_slots_per_gpu(tmp_path)

Verify that gcp spot runner allows multiple reconstruction slots per gpu.

test_gcp_spot_runner_auto_uses_three_reconstruction_slots_on_large_gpu(tmp_path)

Verify that gcp spot runner auto uses three reconstruction slots on large gpu.