时间戳

首页 > 解决方案 > 处理 git clone 错误异常不起作用

python - 处理 git clone 错误异常不起作用

问题描述

我有这个脚本，它克隆项目并从 gitlab.com 获取它的 LFS 对象。除了“例外”部分之外，大部分脚本都可以正常工作。我的一些项目破坏了 LFS 对象，导致克隆过程和脚本失败。我认为最好的处理方法是为它创建一个例外。不幸的是，例外不起作用。无论项目是否失败，都会克隆该项目。当我在“try”中运行克隆命令时，通常在克隆具有损坏的 LFS 对象的项目时发生的错误似乎被忽略了。我从来没有使用过异常，但在我看来，它不起作用的原因是在执行克隆命令之后而不是在命令执行期间发生错误。

import gitlab
import os
import git
from git import Repo
import time
import logging

token = os.environ['TOKEN']

gl = gitlab.Gitlab('https://gitlab.com', private_token=token)
group = gl.groups.get(GROUP_ID) # Group_ID = Any gitlab.com group ID
all_projects = group.projects.list(include_subgroups=True, all=True)
length = len(all_projects)

backup_dir = "/mnt/storage/backup/gitlab-daily"

for each in all_projects:

    project_name = each.attributes['name']
    url = each.ssh_url_to_repo
    project_dir = f"{backup_dir}/{project_name}"
        
    # Get number of commits in the project
    project = gl.projects.get(each.id)
    gitlab_commits = project.commits.list()
    commits_length=len(gitlab_commits)

    # If the project already exists, pull changes. 
    if os.path.exists(project_dir):
        print(f"<{project_name}> already exists. Pulling changes...")
        os.chdir(project_dir)
        git.cmd.Git().pull()
        os.system('git lfs fetch --all > /dev/null')

    # If project doesn't exists and contains at least 1 commit, clone it, but only if the clone 
    # process will not exit with error.
    elif not os.path.exists(project_dir) and commits_length > 0:
        # Try to clone the projects
        try:
            Repo.clone_from(url, project_dir) # try to clone the project
        # If the the clone process returns error, save the error into log file and skip the rest 
        # of the loop.
        except git.GitCommandError:
            logging.basicConfig(filename=(f'{backup_dir}/errors'), format='%(asctime)s:%(message)s', datefmt='%m/%d/%Y', level=logging.ERROR)
            logging.error(f'Project {project_name} contains most probably one or more broken LFS Files!')
            continue
        # If clone process doesn't exit with error continue with this:
        else:            
            print(f"<{project_name}> doesn't exist yet. Cloning...")
            os.system('git lfs fetch --all > /dev/null')

如果我没有发现错误，这就是错误回溯：

    <PROJECT_NAME> doesn't exist yet. Cloning...
Traceback (most recent call last):
  File "/opt/gitlab-daily-backup/./daily-backup.py", line 40, in <module>
    Repo.clone_from(url, project_dir)
  File "/usr/local/lib/python3.9/site-packages/git/repo/base.py", line 1032, in clone_from
    return cls._clone(git, url, to_path, GitCmdObjectDB, progress, multi_options, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/git/repo/base.py", line 973, in _clone
    finalize_process(proc, stderr=stderr)
  File "/usr/local/lib/python3.9/site-packages/git/util.py", line 329, in finalize_process
    proc.wait(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/git/cmd.py", line 408, in wait
    raise GitCommandError(self.args, status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v git@gitlab.com:path/to/project.git /mnt/storage/backup/gitlab-daily/project_name
  stderr: 'Cloning into '/mnt/storage/backup/gitlab-daily/project_name'...
Downloading data/test_images.zip (23 MB)
Error downloading object: data/test_images.zip (3295f51): Smudge error: Error downloading data/test_images.zip (3295f516788842c0ffaf68dd9cb8874d1169fa74b4126087aefe9eab281b9856): [3295f516788842c0ffaf68dd9cb8874d1169fa74b4126087aefe9eab281b9856] Object does not exist on the server or you don't have permissions to access it: [404] Object does not exist on the server or you don't have permissions to access it

Errors logged to /mnt/storage/backup/gitlab-daily/project_name/.git/lfs/logs/20210201T223931.898981748.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: data/test_images.zip: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

标签： pythonpython-3.xexceptiongitlab

解决方案

错误消息显示“警告：克隆成功，但结帐失败。” 因此，如果您想要的最终状态是 project_dir 在这种情况下将不存在，您将不得不在 except 子句中主动删除它。

推荐阅读