my.location.google_takeout fails to load from cache #54

seanbreckenridge · 2023-09-20T02:21:40Z

Not sure if you can reproduce and havent tried to make a minimal example yet, but I thought would at least report the error:

Calling hpi doctor -S my.location.google_takeout fails to load from cache?

I tried clearing the entire cachew dir and just restarting from nothing but same issue

$ hpi --debug doctor -S my.location.google_takeout
[DEBUG   2023-09-19 19:16:21,995 my.google.takeout.parser __init__.py:411 ] [my.google.takeout.parser:events] using inferred type multiple typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]
[DEBUG   2023-09-19 19:16:21,998 my.location.google_takeout __init__.py:411 ] [my.location.google_takeout:locations] using inferred type multiple <class 'my.location.common.Location'>
✅ OK  : my.location.google_takeout                        
[DEBUG   2023-09-19 19:16:22,000 my.location.google_takeout __init__.py:602 ] [my.location.google_takeout:locations] using sqlite:/home/sean/.cache/cachew/my.location.google_takeout:locations for cache
[DEBUG   2023-09-19 19:16:22,001 my.location.google_takeout __init__.py:727 ] [my.location.google_takeout:locations] new hash: {"cachew": "0.14.20230920", "schema": "<class 'my.location.common.Location'>", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:16:22,006 my.location.google_takeout __init__.py:733 ] [my.location.google_takeout:locations] old hash: None
[DEBUG   2023-09-19 19:16:22,006 my.location.google_takeout __init__.py:740 ] [my.location.google_takeout:locations] hash mismatch: computing data and writing to db
[DEBUG   2023-09-19 19:16:22,007 my.google.takeout.parser __init__.py:602 ] [my.google.takeout.parser:events] using sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events for cache
[DEBUG   2023-09-19 19:16:22,008 my.google.takeout.parser __init__.py:727 ] [my.google.takeout.parser:events] new hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:16:22,011 my.google.takeout.parser __init__.py:733 ] [my.google.takeout.parser:events] old hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:16:22,011 my.google.takeout.parser __init__.py:736 ] [my.google.takeout.parser:events] hash matched: loading from cache
[INFO    2023-09-19 19:16:22,129 my.google.takeout.parser __init__.py:707 ] [my.google.takeout.parser:events] loading 1040860 objects from cachew (sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events)
[ERROR   2023-09-19 19:16:24,122 my.location.google_takeout __init__.py:282 ] [my.location.google_takeout:locations] error while setting up cache, falling back to non-cached version
[ERROR   2023-09-19 19:16:24,122 my.location.google_takeout __init__.py:283 ] [my.location.google_takeout:locations] shouldn't happen!
Traceback (most recent call last):
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/__init__.py", line 753, in cachew_wrapper
    yield from written_to_cache()
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/__init__.py", line 694, in written_to_cache
    dct = marshall.dump(obj)
          ^^^^^^^^^^^^^^^^^^
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/marshall/cachew.py", line 39, in dump
    return self.schema.dump(obj)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/marshall/cachew.py", line 98, in dump
    return {
           ^
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/marshall/cachew.py", line 102, in <dictcomp>
    k: ks.dump(getattr(obj, k))
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/marshall/cachew.py", line 137, in dump
    assert False, "shouldn't happen!"
AssertionError: shouldn't happen!
[DEBUG   2023-09-19 19:16:24,123 my.google.takeout.parser __init__.py:602 ] [my.google.takeout.parser:events] using sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events for cache
[DEBUG   2023-09-19 19:16:24,124 my.google.takeout.parser __init__.py:727 ] [my.google.takeout.parser:events] new hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:16:24,126 my.google.takeout.parser __init__.py:733 ] [my.google.takeout.parser:events] old hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:16:24,127 my.google.takeout.parser __init__.py:736 ] [my.google.takeout.parser:events] hash matched: loading from cache
[INFO    2023-09-19 19:16:24,239 my.google.takeout.parser __init__.py:707 ] [my.google.takeout.parser:events] loading 1040860 objects from cachew (sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events)
✅     - stats: {'locations': {'count': 555271, 'last': datetime.datetime(2023, 7, 6, 1, 37, 18, 420000, tzinfo=datetime.timezone.utc)}}
[DEBUG   2023-09-19 19:16:30,617 my.core.structure structure.py:167 ] at exit warning: Found leftover files in temporary directory '[PosixPath('/tmp/HPI-tempdir/tmpam1foziw')]'. this may be because you have multiple hpi processes running -- if so this can be ignored

This is fine though:

$ hpi --debug doctor -S my.google.takeout.parser      
[DEBUG   2023-09-19 19:21:09,560 my.google.takeout.parser __init__.py:411 ] [my.google.takeout.parser:events] using inferred type multiple typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]
✅ OK  : my.google.takeout.parser                          
[DEBUG   2023-09-19 19:21:09,562 my.google.takeout.parser __init__.py:602 ] [my.google.takeout.parser:events] using sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events for cache
[DEBUG   2023-09-19 19:21:09,563 my.google.takeout.parser __init__.py:727 ] [my.google.takeout.parser:events] new hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:21:09,567 my.google.takeout.parser __init__.py:733 ] [my.google.takeout.parser:events] old hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:21:09,567 my.google.takeout.parser __init__.py:736 ] [my.google.takeout.parser:events] hash matched: loading from cache
[INFO    2023-09-19 19:21:09,660 my.google.takeout.parser __init__.py:707 ] [my.google.takeout.parser:events] loading 1040860 objects from cachew (sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events)
✅     - stats: {'events': {'count': 1040860, 'last': datetime.datetime(2020, 1, 4, 7, 24, 19, tzinfo=<DstTzInfo 'US/Pacific' PST-1 day, 16:00:00 STD>)}}

The text was updated successfully, but these errors were encountered:

seanbreckenridge · 2023-09-20T02:23:45Z

Well, nevermind, just removing the file rrm ~/.cache/cachew/my.location.google_takeout:locations and re-running it makes it work?

$ hpi --debug doctor -S my.google.takeout.parser          
[DEBUG   2023-09-19 19:23:06,426 my.google.takeout.parser __init__.py:411 ] [my.google.takeout.parser:events] using inferred type multiple typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]
✅ OK  : my.google.takeout.parser                          
[DEBUG   2023-09-19 19:23:06,432 my.google.takeout.parser __init__.py:602 ] [my.google.takeout.parser:events] using sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events for cache
[DEBUG   2023-09-19 19:23:06,434 my.google.takeout.parser __init__.py:727 ] [my.google.takeout.parser:events] new hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:23:06,445 my.google.takeout.parser __init__.py:733 ] [my.google.takeout.parser:events] old hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:23:06,446 my.google.takeout.parser __init__.py:736 ] [my.google.takeout.parser:events] hash matched: loading from cache
[INFO    2023-09-19 19:23:06,667 my.google.takeout.parser __init__.py:707 ] [my.google.takeout.parser:events] loading 1040860 objects from cachew (sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events)
✅     - stats: {'events': {'count': 1040860, 'last': datetime.datetime(2020, 1, 4, 7, 24, 19, tzinfo=<DstTzInfo 'US/Pacific' PST-1 day, 16:00:00 STD>)}}

Still strange that it couldnt recover though... I ran hpi doctor -S my.location.google_takeout multiple times and it would end up with the same error each time

Wonder if I had some other code with the old cachew writing to the file while this was trying to create it or something... couldve been that

Will leave this open for a bit incase I see the error again (and just good to have it up here), but will close if its not an issue

seanbreckenridge · 2023-09-20T02:30:20Z

Oh -- nevermind again, still an issue; should have checked on warm cache:

 $ hpi --debug doctor -S my.location.google_takeout
[DEBUG   2023-09-19 19:29:49,541 my.google.takeout.parser __init__.py:411 ] [my.google.takeout.parser:events] using inferred type multiple typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]
[DEBUG   2023-09-19 19:29:49,543 my.location.google_takeout __init__.py:411 ] [my.location.google_takeout:locations] using inferred type multiple <class 'my.location.common.Location'>
✅ OK  : my.location.google_takeout                        
[DEBUG   2023-09-19 19:29:49,545 my.location.google_takeout __init__.py:602 ] [my.location.google_takeout:locations] using sqlite:/home/sean/.cache/cachew/my.location.google_takeout:locations for cache
[DEBUG   2023-09-19 19:29:49,545 my.location.google_takeout __init__.py:727 ] [my.location.google_takeout:locations] new hash: {"cachew": "0.14.20230920", "schema": "<class 'my.location.common.Location'>", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:29:49,548 my.location.google_takeout __init__.py:733 ] [my.location.google_takeout:locations] old hash: None
[DEBUG   2023-09-19 19:29:49,549 my.location.google_takeout __init__.py:740 ] [my.location.google_takeout:locations] hash mismatch: computing data and writing to db
[DEBUG   2023-09-19 19:29:49,550 my.google.takeout.parser __init__.py:602 ] [my.google.takeout.parser:events] using sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events for cache
[DEBUG   2023-09-19 19:29:49,550 my.google.takeout.parser __init__.py:727 ] [my.google.takeout.parser:events] new hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:29:49,552 my.google.takeout.parser __init__.py:733 ] [my.google.takeout.parser:events] old hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:29:49,553 my.google.takeout.parser __init__.py:736 ] [my.google.takeout.parser:events] hash matched: loading from cache
[INFO    2023-09-19 19:29:49,617 my.google.takeout.parser __init__.py:707 ] [my.google.takeout.parser:events] loading 1040860 objects from cachew (sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events)
[ERROR   2023-09-19 19:29:50,939 my.location.google_takeout __init__.py:282 ] [my.location.google_takeout:locations] error while setting up cache, falling back to non-cached version
[ERROR   2023-09-19 19:29:50,939 my.location.google_takeout __init__.py:283 ] [my.location.google_takeout:locations] shouldn't happen!
Traceback (most recent call last):
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/__init__.py", line 753, in cachew_wrapper
    yield from written_to_cache()
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/__init__.py", line 694, in written_to_cache
    dct = marshall.dump(obj)
          ^^^^^^^^^^^^^^^^^^
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/marshall/cachew.py", line 39, in dump
    return self.schema.dump(obj)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/marshall/cachew.py", line 98, in dump
    return {
           ^
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/marshall/cachew.py", line 102, in <dictcomp>
    k: ks.dump(getattr(obj, k))
       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sean/.local/lib/python3.11/site-packages/cachew/marshall/cachew.py", line 137, in dump
    assert False, "shouldn't happen!"
AssertionError: shouldn't happen!
[DEBUG   2023-09-19 19:29:50,940 my.google.takeout.parser __init__.py:602 ] [my.google.takeout.parser:events] using sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events for cache
[DEBUG   2023-09-19 19:29:50,941 my.google.takeout.parser __init__.py:727 ] [my.google.takeout.parser:events] new hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:29:50,942 my.google.takeout.parser __init__.py:733 ] [my.google.takeout.parser:events] old hash: {"cachew": "0.14.20230920", "schema": "typing.Union[google_takeout_parser.models.Activity, google_takeout_parser.models.LikedYoutubeVideo, google_takeout_parser.models.PlayStoreAppInstall, google_takeout_parser.models.Location, google_takeout_parser.models.ChromeHistory, google_takeout_parser.models.YoutubeComment, google_takeout_parser.models.PlaceVisit, Exception]", "dependencies": "['google_takeout_version: 0.1.3', '/home/sean/data/google_takeout/Takeout-1599315526.zip', '/home/sean/data/google_takeout/Takeout-1599728222.zip', '/home/sean/data/google_takeout/Takeout-1616796262.zip', '/home/sean/data/google_takeout/Takeout-1634828138.zip', '/home/sean/data/google_takeout/Takeout-1644744478.zip', '/home/sean/data/google_takeout/Takeout-1659510101.zip', '/home/sean/data/google_takeout/Takeout-1667366543.zip', '/home/sean/data/google_takeout/Takeout-1674845469.zip', '/home/sean/data/google_takeout/Takeout-1680326990.zip', '/home/sean/data/google_takeout/Takeout-1688611240.zip']"}
[DEBUG   2023-09-19 19:29:50,943 my.google.takeout.parser __init__.py:736 ] [my.google.takeout.parser:events] hash matched: loading from cache
[INFO    2023-09-19 19:29:51,008 my.google.takeout.parser __init__.py:707 ] [my.google.takeout.parser:events] loading 1040860 objects from cachew (sqlite:/home/sean/.cache/cachew/my.google.takeout.parser:events)

seanbreckenridge · 2023-09-30T21:03:13Z

Ah, looks like this is because of a mismatch between int != float

ipdb> a.type
self = SUnion(type=typing.Optional[float], args=((0, SPrimitive(type=<class 'float'>)), (1, SPrimitive(type=<class 'NoneType'>))))
obj = 10
ipdb> for tidx, a in self.args:
    print(obj, a.type)
 
10 <class 'float'>
10 <class 'NoneType'>
ipdb> obj
10

I was using an int as accuracy in google_takeout_parser, but the location in my.location.common in HPI uses a float, so it works fine in code but doesnt match the signature. seems like this is something mypy doesnt warn for:

def test(x: float) -> float:
    return x * 2


test(4) # with int, not float

Feels like something some could get bit on very easily, not sure if you want to add a special case

karlicoss · 2023-10-01T10:20:41Z

Hmm yeah, I guess it makes sense to allow implicit conversions as long as the underlying serializer (i.e. json in this case) can handle it consistently. I guess it would also break if something passes None without using an optional type, quite likely in runtime and usually not a huge deal (at the very least this shouldn't break cachew)

So I think it's

int can be a float
None can be any type

Can you think of anything else?

seanbreckenridge · 2023-10-01T16:32:32Z

No, I think thats it

Was just checking my autotui code since Ive written similar code like this before, only other think I check for is "true"/"false" and convert it to a bool, but thats more because its user-edited, shouldn't happen with cachew

should resolve #54

seanbreckenridge · 2024-01-29T05:51:06Z

Oh, I was still running into the issue and thought there was some other issue but it looks like the latest commit isnt on pypi

It works if I install from git

Could you do a release so latest commit is included and I can bump the required version on google_takeout_parser

karlicoss · 2024-01-29T21:45:43Z

released!

seanbreckenridge changed the title ~~my.location.google_takeout~~ my.location.google_takeout fails to load from cache Sep 20, 2023

seanbreckenridge mentioned this issue Sep 30, 2023

location: make accuracy default config floats karlicoss/HPI#314

Merged

karlicoss added a commit that referenced this issue Nov 8, 2023

marshall: allow implicit cast of int to float and None to any other type

3b7f9b5

should resolve #54

karlicoss added a commit that referenced this issue Nov 8, 2023

marshall: allow implicit cast of int to float and None to any other type

e9e464c

should resolve #54

karlicoss mentioned this issue Nov 8, 2023

marshall: allow implicit cast of int to float and None to any other type #60

Merged

karlicoss closed this as completed in #60 Nov 8, 2023

karlicoss added a commit that referenced this issue Nov 8, 2023

marshall: allow implicit cast of int to float and None to any other type

a294b26

should resolve #54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

my.location.google_takeout fails to load from cache #54

my.location.google_takeout fails to load from cache #54

seanbreckenridge commented Sep 20, 2023

seanbreckenridge commented Sep 20, 2023 •

edited

Loading

seanbreckenridge commented Sep 20, 2023 •

edited

Loading

seanbreckenridge commented Sep 30, 2023 •

edited

Loading

karlicoss commented Oct 1, 2023 •

edited

Loading

seanbreckenridge commented Oct 1, 2023 •

edited

Loading

seanbreckenridge commented Jan 29, 2024

karlicoss commented Jan 29, 2024

my.location.google_takeout fails to load from cache #54

my.location.google_takeout fails to load from cache #54

Comments

seanbreckenridge commented Sep 20, 2023

seanbreckenridge commented Sep 20, 2023 • edited Loading

seanbreckenridge commented Sep 20, 2023 • edited Loading

seanbreckenridge commented Sep 30, 2023 • edited Loading

karlicoss commented Oct 1, 2023 • edited Loading

seanbreckenridge commented Oct 1, 2023 • edited Loading

seanbreckenridge commented Jan 29, 2024

karlicoss commented Jan 29, 2024

seanbreckenridge commented Sep 20, 2023 •

edited

Loading

seanbreckenridge commented Sep 20, 2023 •

edited

Loading

seanbreckenridge commented Sep 30, 2023 •

edited

Loading

karlicoss commented Oct 1, 2023 •

edited

Loading

seanbreckenridge commented Oct 1, 2023 •

edited

Loading