Skip to content

Add timeout to apt update to prevent indefinite hang on network issues#5577

Merged
dassayantan24 merged 2 commits into
masterfrom
users/dassayantan/prevent-indefinite-hang
May 21, 2026
Merged

Add timeout to apt update to prevent indefinite hang on network issues#5577
dassayantan24 merged 2 commits into
masterfrom
users/dassayantan/prevent-indefinite-hang

Conversation

@dassayantan24
Copy link
Copy Markdown
Contributor

@dassayantan24 dassayantan24 commented May 20, 2026

Context

Self-hosted VMSS agents on Linux hang indefinitely during startup when external Ubuntu repos are unreachable. The installdependencies.sh script runs apt update with no timeout — if the network silently drops packets, the agent never becomes healthy and pipeline jobs are stuck until manual intervention

AB#2393470


Description

#5553


Risk Assessment (Low / Medium / High)

Low


Unit Tests Added or Updated (Yes / No)

No


Additional Testing Performed

  • Manually tested on Linux self-hosted agent where the below command exited with a 2-minute timeout
 docker run --rm -v "$(pwd)/src/Misc/layoutbin:/agent-bin" ubuntu:24.04 \
   bash -c "
     echo 'deb http://192.0.2.1/ubuntu noble main' > /etc/apt/sources.list
     rm -rf /etc/apt/sources.list.d/*
     echo 'Starting at:' && date
     bash /agent-bin/installdependencies.sh
     echo 'Finished at:' && date
     echo 'Exit code:' \$?
   "
  • Additionally, I tested the self-hosted agent on regular pipelines as well.

Change Behind Feature Flag (Yes / No)

No


Tech Design / Approach

  • Design has been written and reviewed.
  • Any architectural decisions, trade-offs, and alternatives are captured.

Documentation Changes Required (Yes/No)

Indicate whether related documentation needs to be updated.

  • User guides, API specs, system diagrams, or runbooks are updated.

Logging Added/Updated (Yes/No)

  • Appropriate log statements are added with meaningful messages.
  • Logging does not expose sensitive data.
  • Log levels are used correctly (e.g., info, warn, error).

Telemetry Added/Updated (Yes/No)

  • Custom telemetry (e.g., counters, timers, error tracking) is added as needed.
  • Events are tagged with proper metadata for filtering and analysis.
  • Telemetry is validated in staging or test environments.

Rollback Scenario and Process (Yes/No)

  • Rollback plan is documented.

Dependency Impact Assessed and Regression Tested (Yes/No)

  • All impacted internal modules, APIs, services, and third-party libraries are analyzed.
  • Results are reviewed and confirmed to not break existing functionality.

@dassayantan24 dassayantan24 changed the title fix: add timeout to apt update to prevent indefinite hang on network … Add timeout to apt update to prevent indefinite hang on network issues May 20, 2026
@dassayantan24 dassayantan24 marked this pull request as ready for review May 20, 2026 12:54
@dassayantan24 dassayantan24 requested review from a team as code owners May 20, 2026 12:54
@dassayantan24 dassayantan24 merged commit ef46f94 into master May 21, 2026
23 checks passed
@dassayantan24 dassayantan24 deleted the users/dassayantan/prevent-indefinite-hang branch May 21, 2026 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants