Piping to ffmpeg throws ERROR failed to run: capacity was less than the current size.

[yettey74]

Hi all,
This is my first project developing in C++.
I have been coding for 30+ years in all sorts.
Tonight I have the following issue when i complete the following steps

# Define repository paths
$repo1 = "c:\project\cpp\gittest"
$repo2 = "c:\project\cpp\gameServer"

# Generate custom log files
gource --output-custom-log log1.txt $repo1
gource --output-custom-log log2.txt $repo2

# Combine the logs
$log1 = Get-Content log1.txt
$log2 = Get-Content log2.txt
$combined = $log1 + $log2

# Sort the combined logs by the timestamp (Convert from UNIX TimeStamp) This could be the error as original is unix timestamp
$sortedCombined = $combined | Sort-Object {
    [datetime]::FromFileTimeUtc((($_.Split('|')[0]) -as [double]) * [timespan]::TicksPerSecond)
}

# Save the sorted logs
$sortedCombined | Set-Content combined.txt

gource combined.txt --auto-skip-seconds 1 --max-files 10000 --output-ppm-stream - | ffmpeg -y -report -r 60 -f image2pipe -vcodec ppm -i pipe: -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 output.mp4

EXAMPLE
combine.txt

1720757821|user1|M|/build/main.pdb
1720757821|user1|A|/build_bak/.ninja_log
1720759202|user1|D|/build_bak/CMakeFiles/main.dir/intermediate.manifest

caption.txt (currently not using in this setup

1720755089|2024-07-12 Not sure what happend
1720755047|2024-07-12 added vscode folder && CMakeLists.txt
1720754665|2024-07-12 first commit

Output from attempt to pipe

ffmpeg version 7.0.1-essentials_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
libavutil 59. 8.100 / 59. 8.100
libavcodec 61. 3.100 / 61. 3.100
libavformat 61. 1.100 / 61. 1.100
libavdevice 61. 1.100 / 61. 1.100
libavfilter 10. 1.100 / 10. 1.100
libswscale 8. 1.100 / 8. 1.100
libswresample 5. 1.100 / 5. 1.100
libpostproc 58. 1.100 / 58. 1.100
Program 'ffmpeg.exe' failed to run: capacity was less than the current size.
Parameter name: requiredLengthAt line:1 char:85

• ... -stream - | ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i pipe: -vcode ...

•             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
  

At line:1 char:1

• gource combined.txt --auto-skip-seconds 1 --max-files 10000 --output- ...

•   + CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
    + FullyQualifiedErrorId : NativeCommandFailed
  

Log file Output

ffmpeg started on 2024-07-13 at 18:01:24
Report written to "ffmpeg-20240713-180124.log"
Log level: 48
Command line:
"C:\ffmpeg\bin\ffmpeg.exe" -y -report -r 60 -f image2pipe -vcodec ppm -i pipe: -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 -probesize 256M -analyzeduration 256M output.mp4
ffmpeg version 7.0.1-essentials_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
libavutil 59. 8.100 / 59. 8.100
libavcodec 61. 3.100 / 61. 3.100
libavformat 61. 1.100 / 61. 1.100
libavdevice 61. 1.100 / 61. 1.100
libavfilter 10. 1.100 / 10. 1.100
libswscale 8. 1.100 / 8. 1.100
libswresample 5. 1.100 / 5. 1.100
libpostproc 58. 1.100 / 58. 1.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-r' ... matched as option 'r' (override input framerate/convert to given output framerate (Hz value, fraction or abbreviation)) with argument '60'.
Reading option '-f' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 'image2pipe'.
Reading option '-vcodec' ... matched as option 'vcodec' (alias for -c:v (select encoder/decoder for video streams)) with argument 'ppm'.
Reading option '-i' ... matched as input url with argument 'pipe:'.
Reading option '-vcodec' ... matched as option 'vcodec' (alias for -c:v (select encoder/decoder for video streams)) with argument 'libx264'.
Reading option '-preset' ... matched as AVOption 'preset' with argument 'ultrafast'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.
Reading option '-crf' ... matched as AVOption 'crf' with argument '1'.
Reading option '-threads' ... matched as AVOption 'threads' with argument '0'.
Reading option '-probesize' ... matched as AVOption 'probesize' with argument '256M'.
Reading option '-analyzeduration' ... matched as AVOption 'analyzeduration' with argument '256M'.
Reading option 'output.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url pipe:.
Applying option r (override input framerate/convert to given output framerate (Hz value, fraction or abbreviation)) with argument 60.
Applying option f (force container format (auto-detected otherwise)) with argument image2pipe.
Applying option vcodec (alias for -c:v (select encoder/decoder for video streams)) with argument ppm.
Successfully parsed a group of options.
Opening an input file: pipe:.
[image2pipe @ 0000022d3b1d2a40] Opening 'pipe:' for reading
[pipe @ 0000022d3b1d2fc0] Setting default whitelist 'crypto,data'

System Setup

OS Name Microsoft Windows 10 Pro
Version 10.0.19045 Build 19045
Other OS Description Not Available
OS Manufacturer Microsoft Corporation
System Name Dev
System Manufacturer Dell Inc.
System Model Precision WorkStation T5500
System Type x64-based PC
System SKU
Processor Intel(R) Xeon(R) CPU E5520 @ 2.27GHz, 2261 Mhz, 4 Core(s), 4 Logical Processor(s)
Processor Intel(R) Xeon(R) CPU E5520 @ 2.27GHz, 2261 Mhz, 4 Core(s), 4 Logical Processor(s)
BIOS Version/Date Dell Inc. A05, 12/04/2010
SMBIOS Version 2.5
Embedded Controller Version 255.255
BIOS Mode Legacy
BaseBoard Manufacturer Dell Inc.
BaseBoard Product 0D883F
BaseBoard Version A06
Platform Role SOHO Server
Secure Boot State Unsupported
PCR7 Configuration Binding Not Possible
Windows Directory C:\Windows
System Directory C:\Windows\system32
Boot Device \Device\HarddiskVolume1
Locale United States
Hardware Abstraction Layer Version = "10.0.19041.3636"
User Name DESKTOP\XXXX
Time Zone Cen. Australia Standard Time
Installed Physical Memory (RAM) 12.0 GB
Total Physical Memory 12.0 GB
Available Physical Memory 4.73 GB
Total Virtual Memory 33.5 GB
Available Virtual Memory 3.87 GB
Page File Space 21.5 GB
Page File C:\pagefile.sys
Kernel DMA Protection Off
Virtualization-based security Not enabled
Device Encryption Support Reasons for failed automatic device encryption: TPM is not usable, PCR7 binding is not supported, Hardware Security Test Interface failed and device is not Modern Standby, Un-allowed DMA capable bus/device(s) detected, Disabled by policy, TPM is not usable
Hyper-V - VM Monitor Mode Extensions Yes
Hyper-V - Second Level Address Translation Extensions Yes
Hyper-V - Virtualization Enabled in Firmware No
Hyper-V - Data Execution Protection Yes

Coding Environment

Visual Studio Code
Visual Studio
Notepad++
VC17
C++23
Python
JS
CMake 3.30

I ahve attempted another way according to the following link here

file structure is

C:.
└───gourceproject
├───repository
| └───.git
| └───.vscode
| └───build
| └───src
| └───version.rc
└───users
| └───user1.jpeg
└───caption.txt
└───gource.config
└───gource.ppm (this is an empty file as it fails during build process piping to ffmpeg)

gource.config file

[[display]
viewport=1920x1080

[gource]
key=true
start-date=2024-07-13
seconds-per-day=1.25
user-image-dir=c:\GourceProject\users
colour-images=true
multi-sampling=1
camera-mode=overview
filename-time=2.0
title=Game Developemnt Experiment
user-scale=1.50
user-font-size=18
stop-at-time=900
hide=mouse,progress
highlight-users=1
highlight-all-users=1
path=C:\GourceProject\repository
output-framerate=60
caption-file=C:\GourceProject\caption.txt
caption-size=22
caption-duration=6

Thsi is where i am up to after a solid couple of hours debuging.
This all started because i wanted to combine my Game Server and Game packages into one and now I am down the rabbithole like rat a up a drain pipe ;-)
I refuse to give in here :-)
Hope to hear from the community 💯 <3

[yettey74]

Update #1
I can run this command until i run out of memory.

gource --load-config gource.config combined.txt --output-ppm-stream - | ffmpeg -y -report -r 60 -f image2pipe -vcodec ppm -i pipe: -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 output.mp4

error

Program 'ffmpeg.exe' failed to run: Exception of type 'System.OutOfMemoryException' was thrown.At
line:1 char:73

Different Conclusion and happy that I am on the right track.