Wiki mirroring implementation

[andrew-boyarshin]

I've implemented wiki mirroring. Admin dashboard operation removed.
Tested on a couple of GitHub repos, with or without wikis.
Any kind of mirrored wiki modification is forbidden.
In this commit I've removed the wiki page name restriction. It now tries correct name, but if it's not found, it tries without space-to-dash replacement. I've done this in order to allow all mirrored wikis to be displayed. You're still not allowed to create or edit page names so that it contains spaces. Any better alternatives?
P.S. I have never written a single line in Go, so coding style might be inconsistent. Applied gofmt corrections.
P.P.S. Meanwhile, thanks for the project! It is a great alternative to GitLab for those, who do not need CI and some advanced features!

[unknwon]

Thanks your PR!

1. Please make separate PRs.
2. How do you determine if a repo has wiki?
3. How do you handle non-github repos?

[andrew-boyarshin]

@unknwon

1. For admin operation? They are closely related (to repo mirroring). Is that necessary?
2. If it succeeds cloning, then yes, it has. In fact, it just clones wiki repo and allows it to fail. wiki repo url is determined by appending .wiki.git to repo url, taking into consideration the possibility of .git suffix existence.
3. I do not. But it works for GitLab and Gogs too. BitBucket uses different scheme, but it can be handled.

[unknwon]

1. For admin operation? They are closely related (to repo mirroring). Is that necessary?

It is.

If it succeeds cloning, then yes, it has. In fact, it just clones wiki repo and allows it to fail. wiki repo url is determined by appending .wiki.git to repo url, taking into consideration the possibility of .git suffix existence.

The error handling is somehow confusion, how to distinguish real fail (not exist) and occasional fail?

1. I do not. But it works for GitLab and gogs too. BitBucket uses different scheme, but it can be handled.

I think we should at least detect URL prefix and only do clone wiki for allowed set of URL prefixes.

[andrew-boyarshin]

@unknwon

1. Ok, will do today.
2. I've looked into your git-module, but didn't find anything that could help me (ls-remote is not implemented), so I will call NewCommand API directly. Will implement real checking with ls-remote.
3. That would limit the use of the feature. What if everything is okay, but the server is not gitlab.com? So I think it would be better to handle bitbucket server wiki url in a different way.

Will implement remote checking and BitBucket workaround right now, then will split into 2 PRs.

[unknwon]

1. That would limit the use of the feature. What if everything is okay, but the server is not gitlab.com? So I think it would be better to handle bitbucket server wiki url in a different way.

It is not an excuse to not check.

[andrew-boyarshin]

@unknwon excuse me, but I can't quite catch what do you mean.
What is preferred, the real existence of wiki in the URL figured out by some code, or the static allowed URL prefixes list?

If the first, then it is exactly what I am going to implement. If the second, we lose the ability to clone any GitLab, Gogs, Bitbucket project not located in their cloud (gitlab.com, gogs.io, bitbucket.org).

[andrew-boyarshin]

@unknwon implemented remote checking, removed any admin dashboard modifications. Will open another PR for dashboard operation.
Working with Gogs, GitHub, GitLab, BitBucket and most likely the majority of other Git services, if they stick to wiki URL scheme like the mentioned ones.
No wiki cloning occurs if wiki not found on the remote. Is it what you wished me to implement?
Edit: bindata still shows in Files changed, but it has no difference with the one in the upstream.

[unknwon]

Thanks, but seems having conflict?

[unknwon]

Any reasons why you add this?

[andrew-boyarshin]

See my comment in PR.

[andrew-boyarshin]

Should we allow user to edit mirrored repos? If we allow, then my IsRepoMirror changes should be reverted. Otherwise we should keep them intact.
The problem of allowing user to edit mirrored repos is syncing with upstream: it overrides any local changes. So what is the point of allowing to edit, if N hours later these changes would be reverted?

[andrew-boyarshin]

@unknwon how about reviewing this PR? It is mergeable now.

[unknwon]

Please sign the CLA.

[andrew-boyarshin]

Done!

[unknwon]

Thanks!

[unknwon]

Now available on https://try.gogs.io/