本文說明如何利用 GitLab API 及 Shell Script 將所有Repositories備份到本機。
2017一開春就遇到開發者圈的大事件 – GitLab DB掛點,影響所及讓全球工程師跳腳,有興趣可以看官方的故障報告,這事件引發幾項後續討論:
- GitLab去年從Microsoft Azure搬到自建機房是否為正確決定? Scalability / performance 之間的衝突要怎麼解決?
- 在雲端運算/儲存已經大行其道的現在,即使大型公司要自建機房也明顯面臨到人才與資源不足的窘境,一旦發生意外都將付出慘痛的代價。
可是以上兩點不在本文討論之列 XD
身為一個用戶,GitLab提供免費帳號無限量的私人Repo,相對GitHub來說具有很大的吸引力,但是一朝被蛇咬,十年怕草繩,在本地存備份,搭配NAS或VPS做鏡像才是正道。
在這邊介紹 dependency 最少的備份方法:Shell Script + GitLab API
取得Access Token
使用GitLab API以前必須先申請Access Token, 可以登入Gitlab.com以後進入 Personal Access Tokens 頁面並將 scopes 設定為 API ,記得千萬要將生成的token存好,離開頁面後就再也無法取得同一個token。
執行程式碼
取得access token以後將其填進 GLAB_TOKEN 變數,執行程式以後就自動會把所有的 repositories都存到 gitlab-backup 目錄底下,還等什麼? 馬上備份你自己寶貴的專案吧!
#!/bin/bash # A script to backup GitLab repositories. GLAB_BACKUP_DIR=${GLAB_BACKUP_DIR-"gitlab_backup"} # where to place the backup files GLAB_TOKEN=${GLAB_TOKEN-"YOUR_TOKEN"} # the access token of the account GLAB_GITHOST=${GLAB_GITHOST-"gitlab.com"} # the GitLab hostname GLAB_PRUNE_OLD=${GLAB_PRUNE_OLD-true} # when `true`, old backups will be deleted GLAB_PRUNE_AFTER_N_DAYS=${GLAB_PRUNE_AFTER_N_DAYS-7} # the min age (in days) of backup files to delete GLAB_SILENT=${GLAB_SILENT-false} # when `true`, only show error messages GLAB_API=${GLAB_API-"https://gitlab.com/api/v3"} # base URI for the GitLab API GLAB_GIT_CLONE_CMD="git clone --quiet --mirror git@${GLAB_GITHOST}:" # base command to use to clone GitLab repos TSTAMP=`date "+%Y%m%d"` # The function `check` will exit the script if the given command fails. function check { "$@" status=$? if [ $status -ne 0 ]; then echo "ERROR: Encountered error (${status}) while running the following:" >&2 echo " $@" >&2 echo " (at line ${BASH_LINENO[0]} of file $0.)" >&2 echo " Aborting." >&2 exit $status fi } # The function `tgz` will create a gzipped tar archive of the specified file ($1) and then remove the original function tgz { check tar zcf $1.tar.gz $1 && check rm -rf $1 } $GLAB_SILENT || (echo "" && echo "=== INITIALIZING ===" && echo "") $GLAB_SILENT || echo "Using backup directory $GLAB_BACKUP_DIR" check mkdir -p $GLAB_BACKUP_DIR $GLAB_SILENT || echo -n "Fetching list of repositories ..." GLAB_PROJ_API="${GLAB_API}/projects?private_token=${GLAB_TOKEN}&per_page=100&simple=true" echo ${GLAB_PROJ_API} REPOLIST=`check curl --silent ${GLAB_PROJ_API} | check perl -p -e "s/,/\n/g" | check grep "\"path_with_namespace\"" | check awk -F':"' '{print $2}' | check sed -e 's/"}//g'` $GLAB_SILENT || echo "found `echo $REPOLIST | wc -w` repositories." $GLAB_SILENT || (echo "" && echo "=== BACKING UP ===" && echo "") for REPO in $REPOLIST; do $GLAB_SILENT || echo "Backing up ${REPO}" check ${GLAB_GIT_CLONE_CMD}${REPO}.git ${GLAB_BACKUP_DIR}/${GLAB_ORG}-${REPO}-${TSTAMP}.git && tgz ${GLAB_BACKUP_DIR}/${GLAB_ORG}-${REPO}-${TSTAMP}.git done if $GLAB_PRUNE_OLD; then $GLAB_SILENT || (echo "" && echo "=== PRUNING ===" && echo "") $GLAB_SILENT || echo "Pruning backup files ${GLAB_PRUNE_AFTER_N_DAYS} days old or older." $GLAB_SILENT || echo "Found `find $GLAB_BACKUP_DIR -name '*.tar.gz' -mtime +$GLAB_PRUNE_AFTER_N_DAYS | wc -l` files to prune." find $GLAB_BACKUP_DIR -name '*.tar.gz' -mtime +$GLAB_PRUNE_AFTER_N_DAYS -exec rm -fv {} > /dev/null \; fi $GLAB_SILENT || (echo "" && echo "=== DONE ===" && echo "") $GLAB_SILENT || (echo "GitLab backup completed." && echo "")</pre> <pre>