Skip to content

ci: add pip/pre-commit cache and fix mirror override in NPU CI #9320

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JavaPythonAIForBAT wants to merge 1 commit into
base: main
Choose a base branch
from
+22 −7
Open

ci: add pip/pre-commit cache and fix mirror override in NPU CI #9320

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .dev_scripts/ci_container_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
if [ "$MODELSCOPE_SDK_DEBUG" == "True" ]; then
# pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
pip install -r requirements/tests.txt -i https://mirrors.aliyun.com/pypi/simple/
pip install -r requirements/tests.txt
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot May 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It is recommended to add error handling to the pip install command. Without it, if the installation of test dependencies fails, the script will continue to subsequent steps (like pre-commit), which may lead to misleading failures or unreliable test results in CI.

Suggested change
pip install -r requirements/tests.txt
pip install -r requirements/tests.txt || exit 1

git config --global --add safe.directory /ms-swift
git config --global user.email tmp
git config --global user.name tmp.com
Expand All @@ -20,8 +20,8 @@ if [ "$MODELSCOPE_SDK_DEBUG" == "True" ]; then
fi
fi

pip install -r requirements/framework.txt -U -i https://mirrors.aliyun.com/pypi/simple/
pip install decord einops -U -i https://mirrors.aliyun.com/pypi/simple/
pip install -r requirements/framework.txt -U
pip install decord einops -U
Comment on lines +23 to +24
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot May 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

These two pip install commands can be combined into a single invocation. This is more efficient as it reduces the overhead of starting the pip process multiple times and allows the dependency resolver to handle all packages together, ensuring consistency. Additionally, adding || exit 1 ensures the script terminates immediately if the installation fails.

Suggested change
pip install -r requirements/framework.txt -U
pip install decord einops -U
pip install -r requirements/framework.txt decord einops -U || exit 1

pip uninstall autoawq -y
pip install optimum
pip install diffusers
Expand All @@ -30,7 +30,7 @@ if [ "$MODELSCOPE_SDK_DEBUG" == "True" ]; then

# test with install
pip install .
pip install auto_gptq bitsandbytes deepspeed -U -i https://mirrors.aliyun.com/pypi/simple/
pip install auto_gptq bitsandbytes deepspeed -U
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot May 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Adding error handling here ensures that the script does not proceed to run the tests if these dependencies fail to install, which helps in identifying environment setup issues early and prevents running tests in an incomplete environment.

Suggested change
pip install auto_gptq bitsandbytes deepspeed -U
pip install auto_gptq bitsandbytes deepspeed -U || exit 1

else
echo "Running case in release image, run case directly!"
fi
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/citest_npu.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,17 @@ jobs:
pip config set global.trusted-host cache-service.nginx-pypi-cache.svc.cluster.local

- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
fetch-depth: ${{ github.event_name == 'pull_request' && 2 || 0 }}

- name: Cache pip packages
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-${{ runner.arch }}-pip-${{ hashFiles('requirements/framework.txt', 'requirements/tests.txt') }}
restore-keys: ${{ runner.os }}-${{ runner.arch }}-pip-

- name: Get changed files
id: changed-files
run: |
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/lint.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,18 @@ jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Set up Python 3.10
uses: actions/setup-python@v2
uses: actions/setup-python@v5
with:
python-version: '3.10'
cache: pip
- name: Cache pre-commit environments
uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}
restore-keys: pre-commit-${{ runner.os }}-
- name: Install pre-commit hook
run: |
pip install pre-commit
Expand Down
Loading