fix: not able to update local created non-urlencoded wiki pages

[BLumia]

Related to issue #16122, this patch intended to resolve part of the bug mentioned in #16122 (comment)

If we clone the wiki repo locally and create some markdown file without urlencoded its filename, the file can be displayed correctly if we then push back it to the remote repo, but if we attempt to update these pages, it will create a new urlencoded file with updated content and the old non-urlencoded file is still here with content unchanged. This PR intended to resolve that by doing:

1. When trying to create/update a wiki page, check if there is a non-urlencoded page that already exists.
2. If such a page not exists, use urlencoded file name instead (keep the original behavior).

---

By the way, since resolving #16122 completely still need some other patches, there is something we still need to discuss:

1. To resolve that issue, do we need to switch to use non-urlencoded files by default?
2. We probably need to keep compatible with the old behavior, right?

If necessary, we can discuss these in #16122.

[codecov-commenter]

Codecov Report

Merging #16139 (ef77ebe) into main (290f458) will increase coverage by 0.02%.
The diff coverage is 65.38%.

@@            Coverage Diff             @@
##             main   #16139      +/-   ##
==========================================
+ Coverage   45.42%   45.44%   +0.02%     
==========================================
  Files         709      709              
  Lines       83611    83625      +14     
==========================================
+ Hits        37978    38002      +24     
+ Misses      39545    39534      -11     
- Partials     6088     6089       +1     

Impacted Files	Coverage Δ
services/wiki/wiki.go	59.33% <65.38%> (+1.38%)	⬆️
services/pull/temp_repo.go	26.59% <0.00%> (-3.20%)	⬇️
services/pull/check.go	25.51% <0.00%> (-2.76%)	⬇️
services/pull/patch.go	54.23% <0.00%> (-1.70%)	⬇️
routers/api/v1/repo/pull.go	28.74% <0.00%> (+0.59%)	⬆️
modules/log/event.go	60.84% <0.00%> (+0.94%)	⬆️
modules/notification/mail/mail.go	38.23% <0.00%> (+1.96%)	⬆️
modules/queue/workerpool.go	56.10% <0.00%> (+2.29%)	⬆️
modules/queue/manager.go	68.36% <0.00%> (+2.82%)	⬆️
services/mailer/mail_comment.go	85.18% <0.00%> (+7.40%)	⬆️
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 290f458...ef77ebe. Read the comment docs.

[BLumia]

To manually test if it's working, do the following steps:

1. Clone the wiki repo locally.
2. Create a new file with the name "测试.md" (without quote), empty content is okay.
3. Add, commit and push that file to the wiki repo.
4. Edit that page online, and submit the change.
5. Pull the wiki repo and see if the file name is changed or if there is a new file created. It will not change the file name nor create any new file after this patch.

While trying to add some test cases I found there is also the same issue in the test code.

gitea/routers/web/repo/wiki_test.go

Line 34 in 5c2f9b3

if entry.Name() == wiki_service.NameToFilename(wikiName) {

Since I'm also not very familiar with the whole Gitea codebase and how these tests works, I think it's better to not add a test case for this PR. If test code is necessary, I hope someone can write this as an example since it can take me a much longer time than others to figure out how to write it correctly. There are some other cases that can be tested and I can then take the test code as a reference to add other test cases.

[techknowlogick]

FWIW if I were to create a file in a Git repo with filename of 测试.md it would keep that as the filename on disk, so I think for the wiki aligning that with how the git repo does it is ok.

[BLumia]

FWIW if I were to create a file in a Git repo with filename of 测试.md it would keep that as the filename on disk, so I think for the wiki aligning that with how the git repo does it is ok.

I agree since it's also what GitHub wiki does and it also makes local management possible, but if we switch to only support to use the filename as-is, it may cause a problem to the existing wiki repo which already contains a lot of pages with urlencoded filenames.

This patch keeps the old behavior to keep compatible with the old Gitea installation with the existing repos. Old wiki repo will still work and people who import their wiki from GitHub or prefer to create files locally will also be able to edit their wiki without issue.

So I guess what we need to decide first is, do we need to keep compatible with the old behavior? If we do, I guess this patch is probably okay, and we can do breaking changes in a future major release.

[zeripath]

Ignore me I'm getting confused

We need to make:

	isWikiExist, newWikiPath, err := prepareWikiFileName(gitRepo, newWikiName)
	if err != nil {
		return err
	}

	if isNew {
		if isWikiExist {
			return models.ErrWikiAlreadyExist{
				Title: newWikiPath,
			}
		}
	} else {
		// avoid check existence again if wiki name is not changed since gitRepo.LsFiles(...) is not free.
		isOldWikiExist := true
		oldWikiPath := newWikiPath
		if oldWikiName != newWikiName {
			isOldWikiExist, oldWikiPath, err = prepareWikiFileName(gitRepo, oldWikiName)
			if err != nil {
				return err
			}
		}

		if isOldWikiExist {
			err := gitRepo.RemoveFilesFromIndex(oldWikiPath)
			if err != nil {
				log.Error("%v", err)
				return err
			}
		}
	}

make sense with the changes in prepareWikiFileName

[BLumia]

I'm not quite sure why that doesn't make sense. Would you mind being more specific?

If you mean the code inside the else block, users may rename the wiki page while editing so we still need to check the old wiki page file name if that's the case.

Thanks!

I'm an idiot and got confused

[zeripath]

This is fine

[techknowlogick]

Thanks :)

[techknowlogick]

🚀

[BLumia]

Thanks!

For people who already have wiki that contains lots of url-encoded files, you can use the following script to rename these files :)

import os
import urllib.parse as urldecode
files = [f for f in os.listdir('.') if os.path.isfile(f)]
for f in files:
    new_name = urldecode.unquote(f)
    if new_name == f:
        print("Skipped: " + f)
        continue
    new_name = new_name.replace("-", " ")
    try:
        os.rename(f, new_name)
        print("Renamed: " + f + " -> " + new_name)
    except OSError:
        print("Failed: " + f + " -> " + new_name)

input("Press Enter to continue...")