Git sync is a tool that I have written to keep my custom hosted git repo in sync with multiple different sources. It can pull from places such as github at a set interval as well as push to any git, http, https, or ssh repository. Additinally I have added some features for populating metadata on cgit repositories that you can turn on if you want.
This works on all OS but hot reloading of config on windows doesn't work.
Here is an example config file with secrets replaced. To validate your config you can
run git-sync -validate -config file.yaml
.
---
interval: 3600
path: /var/lib/repos
repos:
- url: https://review.coreboot.org/coreboot.git
type: fetch_mirror
path: /var/lib/repos/mirrors/coreboot
extras:
cgitowner: coreboot
cgitsection: mirrors
metadata:
- cgit
- url: https://anongit.gentoo.org/git/repo/gentoo.git
type: fetch_mirror
path: /var/lib/repos/mirrors/gentoo
extras:
cgitowner: gentoo
cgitsection: mirrors
metadata:
- cgit
- url: https://github.com/michaeljs1990/collins-go-cli.git
type: push_mirror
path: /var/lib/repos/xrt0x/collins-go-cli
httpauth:
user: michaeljs1990
token: some-token
remote: github_mirror
github:
- username: michaeljs1990
httpauth:
user: michaeljs1990
token: some-token
repos: true
metadata:
- cgit
push_mirror
- Push from the local path provided to the given URL to keep them in sync.
fetch_mirror
- Pull from the url provided and place it at the provided path.
github
- pull all repos located under the provided username to the top level path.
Run one time to test out your config
git-sync -config config.yaml -verbose -oneshot
Start the proces as a long running service
git-sync -config config.yaml -log_file git_sync.log
Log to file as JSON
git-sync -config config.yaml -log_file git_sync.log -log_format json
Run multiple workers to speed things up
git-sync -config config.yaml -log_file git_sync.log -log_format json -workers 10
Validate your config
git-sync -validate
You can reload the config file for your git-sync process with the use of usr1. This will validate the config before doing the reload so if you have a bad config it will just output an error but keep running with the old config.
If you are excited to have something pushed out or want a new update you can hit git-sync with usr2. This will cause it to sync all repositories again that it knows about.
Test with golang 1.11 - 1.16
Feature | Done | Kinda | Planned | Wishful |
---|---|---|---|---|
Clone repositories from bitbucket | X | |||
Clone repositories from gitlab | X | |||
Add unit tests | X | |||
Add file watching for push repos | X | |||
Clone repositories from github | X | |||
Push repo over http auth | X | |||
Pull repo over http auth | X | |||
Push repo over git protocol | X | |||
Pull repo over git protocol | X | |||
Push repo over SSH protocol | X | |||
SSH user/pass auth | X | |||
SSH private key auth | X | |||
Pull repo over SSH protocol | X | |||
Config validation | X | |||
Prometheus metrics | X | |||
Hot config reloading | X | |||
Force syncing without process restart | X | |||
Populate some cgit metadata | X |