export with original_filename can result in incorrect results if exported to same folder #134
Comments
|
I'm experiencing this issue. I have multiple files in iPhotos named Behaviour on WindowsMy primary computer is a Windows-based device, which has the iCloud applications installed. ❯ find /mnt/c/Users/username/iCloudPhotos/Photos -iname 'FullSizeRender*' -exec md5sum {} \; | sort
05f0f5d2f82700db2d9f211806b2a08d /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(10).jpeg
06b13e0f6ffe98d9c1635a761c28a366 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(15).jpeg
180d9823cd919c2b727741fc55fb9362 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(20).jpeg
1ad75b1a37d6b437c6547fa597313e57 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(7).jpeg
3dedac37a9aad0837ec88fe7c44420de /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(17).jpeg
42490dcb581d6dea7366a8e270e5c28c /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender.jpeg
48e386b1bfd7c48f4d96c3d23d2ac5b0 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender.heic
501f054ab9a16d9441aec74c0e78dbe5 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(3).jpeg
6816c56416a3381b9306c2c896775a5a /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(5).jpeg
69b831ad0c9ba2c3e5347d70150184c4 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(4).jpeg
6a75c23a7af644752975912ac317144f /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(1).heic
6fa7e4470dc8920540b73df51946db33 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(16).jpeg
752014bf99130645b433656a62e91262 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(23).jpeg
77a7226d6f2b60aa5753c4ed7f2d77f9 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender.mov
8c46a06c0ff993122831cb753a7a176b /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(8).jpeg
8f46829a5067c932fdd7e5563029a287 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(21).jpeg
a1598ccff016ed5606c129fb3170d61a /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(24).jpeg
ad77b8a58e222b2e219042023af6b64b /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(14).jpeg
b22d19a50663e5216f099d5bf3011cfd /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(25).jpeg
b70d80401408f8886a7aa49539c15cda /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(12).jpeg
ba8e86234d41e6fb3556c1e8e20d643a /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(6).jpeg
baf20cc80c9c93ec8c7af6763ab8da54 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(2).jpeg
c16c8f101d5679da8b306f5628b7ec12 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(18).jpeg
ca3ca2d974c97ed37154d02986b15e07 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(3).heic
d2d12fc7fdf648733b187ba035bc2279 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(9).jpeg
dde3bd5fa67c64eef887cb986b80d28a /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(13).jpeg
e9481482b60cb050d3b75c6831deaa8e /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(26).jpeg
f873599c08280877d992709c00c290f6 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(2).heic
fbcf0e4fd4e5fae7ea36befa07f514c7 /mnt/c/Users/username/iCloudPhotos/Photos/FullSizeRender(1).jpegBehaviour in OSX PhotosMy setup:
My process:
It looks to me like:
Run 1The files produced by run 1: ba8e86234d41e6fb3556c1e8e20d643a /mnt/d/photos/FullSizeRender (1) (1) (1) (1).jpeg
adaf4499936599d5f5539386f144f365 /mnt/d/photos/FullSizeRender (1) (1) (1) (1).jpeg.json
d25c6440c4bdaa9bf2477ebc35a0f960 /mnt/d/photos/FullSizeRender (1) (1) (1) (1).mov
a1598ccff016ed5606c129fb3170d61a /mnt/d/photos/FullSizeRender (1) (1) (1) (2).jpeg
0924cbe470f31e1ac8777c6b95e6e2e4 /mnt/d/photos/FullSizeRender (1) (1) (1) (2).jpeg.json
fe2167415692e162b87f316693c036d6 /mnt/d/photos/FullSizeRender (1) (1) (1) (2).movReduced logs from run 1, where Run 2The files produced by run 2: a1598ccff016ed5606c129fb3170d61a /mnt/d/photos/FullSizeRender (1) (1) (1) (3).jpeg
585b033596f273f974ce82fb6628d94a /mnt/d/photos/FullSizeRender (1) (1) (1) (3).jpeg.json
fe2167415692e162b87f316693c036d6 /mnt/d/photos/FullSizeRender (1) (1) (1) (3).mov
ba8e86234d41e6fb3556c1e8e20d643a /mnt/d/photos/FullSizeRender (1) (1) (1) (4).jpeg
83a1bec523df19ec8aaf5502cd208a26 /mnt/d/photos/FullSizeRender (1) (1) (1) (4).jpeg.json
d25c6440c4bdaa9bf2477ebc35a0f960 /mnt/d/photos/FullSizeRender (1) (1) (1) (4).movReduced logs from run 2, where Run 3The files produced by run 3: a1598ccff016ed5606c129fb3170d61a /mnt/d/photos/FullSizeRender (1) (1) (1) (1) (1) (1).jpeg
6b24a5739fafb82fe21693ebdecc234a /mnt/d/photos/FullSizeRender (1) (1) (1) (1) (1) (1).jpeg.json
fe2167415692e162b87f316693c036d6 /mnt/d/photos/FullSizeRender (1) (1) (1) (1) (1) (1).mov
ba8e86234d41e6fb3556c1e8e20d643a /mnt/d/photos/FullSizeRender (1) (1) (1) (1) (1) (2).jpeg
5de9cd812ed637b9ca47eaaec372e7c2 /mnt/d/photos/FullSizeRender (1) (1) (1) (1) (1) (2).jpeg.json
d25c6440c4bdaa9bf2477ebc35a0f960 /mnt/d/photos/FullSizeRender (1) (1) (1) (1) (1) (2).movReduced logs from run 3, where |
|
@teh-hippo I tried with a test library to replicate this behavior but osxphotos works as expected. This may have something to do with the target volume. What OS are you running the osxphotos command on? What format is the target volume? Is it a NAS or an external disk? I'll try to think of some debug options to give you to determine why the filename is incremented incorrectly. In my testing with 3 identically named files and your command line options, osxphotos behaves as expected. |
|
@teh-hippo try running export as follows: Select a couple of the
This will enable debugging on the functions that determine the filename so hopefully I can figure out what's going. Post output here or send to me at osxphotos.py@gmail.com |
|
This is getting weird mate. I can't replicate it either. I tried with However, it does still break on the full run. I've sent debug logs from those runs to the email provided. It's very theoretical, but I can't work it out either. Please review, and let me know if there's more than I could run to help out here. |
|
@teh-hippo meanwhile may you shed some light on the following ?
And are your volumes/file systems "case sensitive" (origin and destination)? |
That may be part or all of the issue. OSXPhotos assumes the target volume is case-insensitive as this is the default for macOS. I do not do any testing with case sensitive file systems. I'll take a look at trying to set up a test environment for this case. |
|
Yourself @RhetTbull & @oPromessa have both been fantastic. Thanks for working through this. |
|
I've done some further testing. Ran the following, which exported the files to main drive. This test was successful. mkdir ~/Test
osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --name FullSizeRender.jpeg ~/test | tee /Volumes/hostshare/on-mac-0-initial.txt
# Processed: 33 photos, exported: 62, updated: 0, skipped: 0, updated EXIF data: 62, missing: 0, error: 0, touched date: 95
osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --name FullSizeRender.jpeg ~/test | tee /Volumes/hostshare/on-mac-1-update.txt
# Processed: 33 photos, exported: 0, updated: 0, skipped: 62, updated EXIF data: 0, missing: 0, error: 0, touched date: 0Ran the following, which exported the files to main drive. This test was also successful. (Minor, I need to use sudo to export - but its my laziness & not related). mkdir /Volumes/hostshare/Test
sudo osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --name FullSizeRender.jpeg /Volumes/hostshare/Test | tee /Volumes/hostshare/ext-0-initial.txt
# Processed: 33 photos, exported: 62, updated: 0, skipped: 0, updated EXIF data: 62, missing: 0, error: 0, touched date: 95
sudo osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --name FullSizeRender.jpeg /Volumes/hostshare/Test | tee /Volumes/hostshare/ext-1-update.txt
# Processed: 33 photos, exported: 0, updated: 0, skipped: 62, updated EXIF data: 0, missing: 0, error: 0, touched date: 0Of note, is that when I compared each of the files ( |
|
I'm going to try a few more tests locally exporting a broader set. I feel like potentially the library was put into a funky state, but I don't know how. Its still behaving as I described above if I export on the main library. |
|
@RhetTbull, I think its something in my library. Let me know how I can assist. Under the assumption that the issue is the drive I'm saving things to, I ran the below script. It exports all favourite images from my library to a location on the Mac 3 times - which should be 1 export, and 2 skips. I then did the same test on the external location. The mac had the same issue as the external drive.
#!/bin/zsh
dname=Fav
echo "Running a test for: ${dname}"
[[ -d "/Users/USERNAME/${dname}" ]] && rm -R "/Users/USERNAME/${dname}"
mkdir "/Users/USERNAME/${dname}"
[[ -d "/Volumes/hostshare/${dname}" ]] && rm -R /Volumes/hostshare/${dname}
mkdir /Volumes/hostshare/${dname}
echo "Running local mac export"
osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --favorite /Users/USERNAME/${dname} > /Volumes/hostshare/${dname}-mac-0-initial.txt
echo "Test 2"
osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --favorite /Users/USERNAME/${dname} > /Volumes/hostshare/${dname}-mac-1-update.txt
echo "Test 3"
osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --favorite /Users/USERNAME/${dname} > /Volumes/hostshare/${dname}-mac-2-update.txt
echo "Running remote export"
osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --favorite /Volumes/hostshare/${dname} > /Volumes/hostshare/${dname}-ext-0-initial.txt
echo "Test 2"
osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --favorite /Volumes/hostshare/${dname} > /Volumes/hostshare/${dname}-ext-1-update.txt
echo "Test 3"
osxphotos export --verbose --only-photos --touch-file --exiftool --update-errors --sidecar exiftool --update --cleanup --favorite /Volumes/hostshare/${dname} > /Volumes/hostshare/${dname}-ext-2-update.txtI'm going to re-run the test, with the |
|
Result with debug logs.
|
|
@teh-hippo thanks for the additional data. The logs appear to show issue #872 & #1619 not this issue. (files getting re-exported due to exiftool errors). I have a fix for this in work (done but I'm working on the tests). Issue #134 (this issue) I've still not been able to replicate but won't have time to really explore much until I finish #1619. |
If an image has the same original_filename as another image and is exported to the same directory,
osxphotos exportwill not detect that it's a different file which will result in incorrect behavior (one of the files will be overwritten if --overwrite or --update is used)This can happen for example if file is duplicated in Photos
The text was updated successfully, but these errors were encountered: