Added 'save-metrics' option #5341
Conversation
* Added 'save-metrics' option # save the metrics to *.txt
|
@glenn-jocher |
|
@SpongeBab thanks for the PR! This is actually a great idea, but I think we'd want the output to be a csv, similar to results.csv, i.e. a val.csv so that the users know what the columns mean. I don't think we need an argument for this, we can just generate this automatically if |
|
@SpongeBab an example csv is here, first row is column names: yolov5/utils/loggers/__init__.py Lines 113 to 119 in a4fece8 |
|
@glenn-jocher yeah, I also think we should generate this automatically if plots is true. How about this? And then I found that you use the callback function |
|
@SpongeBab ah! Ok I finally understand now what we need. CSV has comma separations by the way, but now I realized we don't want CSV, we just want to log the console output to a file. I found a link that can help do this: Though this will also require updates to the loggers pending in #4854 The idea is that anything that gets logged to console (everything you see on your screen) during # create logger with 'spam_application'
logger = logging.getLogger('spam_application')
logger.setLevel(logging.DEBUG)
# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)EDIT: This is really exciting, we can do this for all our main files that create directories in run/, i.e. train.py, val.py, detect.py. This will allow all users to see the exact console output that created those files/results. |
|
yeahοΌthat's right! |
|
@SpongeBab yes I need to update #4854 and merge that first. I'll try to get that done soon, and then we can update this PR to create log files by default for all the main functions. |
|
@SpongeBab Loggers update is merged now in PR #4854. Every file has a print('message') # old
LOGGER.info('message') # newThis should pave the way to use the stackoverflow method in #5341 (comment) to add text logging to |
|
@glenn-jocher , I tried https://stackoverflow.com/questions/6386698/how-to-write-to-a-file-using-the-logging-python-module. So then I put it under And then I put these code in here: EDIT: Then I can do the same work on the train,py and detect.py ie. |
|
@SpongeBab thanks, I see the updates! Not sure what to do about the color string coding, will search for solutions. |
for more information, see https://pre-commit.ci
|
@glenn-jocher Hi,I have a question. Take it your way~ |
|
@SpongeBab
|
|
@glenn-jocher Hh, One line just.Yeah, I did this in the beginning, while I need to use This is the output: The process cannot access the file beacause it is being used by another process. Report to you. |
|
And report a small bug. |
|
@SpongeBab this is interesting. By complete coincidence I just fixed EDIT: the increment_paths() fix allows it to make new directories containing decimals now. This is the only change. If you increment a directory with decimals it will increment it as a file: path/to/dir.abc.def
path/to/dir.abc2.def # increment decimal directory
path/to/dir.abc3.def # increment decimal directory (I should probably fix so number is at end) |
|
TODO:
|
Resolves #5341 (comment) Tests: ```python import glob import re from pathlib import Path def increment_path(path, exist_ok=False, sep='', mkdir=False): # Increment file or directory path, i.e. runs/exp --> runs/exp{sep}2, runs/exp{sep}3, ... etc. path = Path(path) # os-agnostic if path.exists() and not exist_ok: path, suffix = (path.with_suffix(''), path.suffix) if path.is_file() else (path, '') dirs = glob.glob(f"{path}{sep}*") # similar paths matches = [re.search(rf"%s{sep}(\d+)" % path.stem, d) for d in dirs] i = [int(m.groups()[0]) for m in matches if m] # indices n = max(i) + 1 if i else 2 # increment number path = Path(f"{path}{sep}{n}{suffix}") # increment path if mkdir: path.mkdir(parents=True, exist_ok=True) # make directory return path print(increment_path('runs')) print(increment_path('export.py')) print(increment_path('abc.def.dir')) print(increment_path('abc.def.file')) ```
|
@glenn-jocher Oh, you closed it, Sorry for that, I can't solve the two problems. |
|
@SpongeBab ah sorry, didn't mean to close. |
|
@SpongeBab sorry, this keeps getting closed for some reason. Did you find any good methods to log to text within save_dir? I think we can ignore the color errors. It's possible they might actually be read in color by the appropriate editor. |
|
/rebase |
|
@glenn-jocher yes,I have looked for some ways to record color string. I tried to search for plug-ins that can identify color string in pycharm. I did find it. But this kind of plugins only support:
I have not found a way to save the color string codes you are using. As for [WinError 32], unless I save it in the root path. Otherwise, will encounter this error. |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions YOLOv5 π and Vision AI β. |
Resolves ultralytics#5341 (comment) Tests: ```python import glob import re from pathlib import Path def increment_path(path, exist_ok=False, sep='', mkdir=False): # Increment file or directory path, i.e. runs/exp --> runs/exp{sep}2, runs/exp{sep}3, ... etc. path = Path(path) # os-agnostic if path.exists() and not exist_ok: path, suffix = (path.with_suffix(''), path.suffix) if path.is_file() else (path, '') dirs = glob.glob(f"{path}{sep}*") # similar paths matches = [re.search(rf"%s{sep}(\d+)" % path.stem, d) for d in dirs] i = [int(m.groups()[0]) for m in matches if m] # indices n = max(i) + 1 if i else 2 # increment number path = Path(f"{path}{sep}{n}{suffix}") # increment path if mkdir: path.mkdir(parents=True, exist_ok=True) # make directory return path print(increment_path('runs')) print(increment_path('export.py')) print(increment_path('abc.def.dir')) print(increment_path('abc.def.file')) ```
Resolves ultralytics/yolov5#5341 (comment) Tests: ```python import glob import re from pathlib import Path def increment_path(path, exist_ok=False, sep='', mkdir=False): # Increment file or directory path, i.e. runs/exp --> runs/exp{sep}2, runs/exp{sep}3, ... etc. path = Path(path) # os-agnostic if path.exists() and not exist_ok: path, suffix = (path.with_suffix(''), path.suffix) if path.is_file() else (path, '') dirs = glob.glob(f"{path}{sep}*") # similar paths matches = [re.search(rf"%s{sep}(\d+)" % path.stem, d) for d in dirs] i = [int(m.groups()[0]) for m in matches if m] # indices n = max(i) + 1 if i else 2 # increment number path = Path(f"{path}{sep}{n}{suffix}") # increment path if mkdir: path.mkdir(parents=True, exist_ok=True) # make directory return path print(increment_path('runs')) print(increment_path('export.py')) print(increment_path('abc.def.dir')) print(increment_path('abc.def.file')) ```
Added 'save-metrics' option(default is false):
π οΈ PR Summary
Made with β€οΈ by Ultralytics Actions
π Summary
Enhanced logging and performance metrics in YOLOv5 validation script.
π Key Changes
--save-metricsflag to save performance metrics to a text file.π― Purpose & Impact
These updates could provide a smoother experience for developers working on model validation and make it easier to monitor and analyze YOLOv5's performance.