Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The library size is very large #57

Open
denesbartha opened this issue Oct 27, 2023 · 19 comments
Open

The library size is very large #57

denesbartha opened this issue Oct 27, 2023 · 19 comments

Comments

@denesbartha
Copy link

I would like to use the library pytorch_lite for object detection using a YOLO8 model.

However I saw that the size of the APK file is too large, and then I found that the file libpytorch_jni.so is ~74MB.
It does not matter whether I specify the version explicitly pytorch_lite: ^4.2.1 or I provide the path: ../ to the pytorch_lite package, the result is roughly the same.

Is there a way to shrink the lib size somehow by removing some unnecessary dependencies?
@abdelaziz-mahdy
Copy link
Owner

Sadly I couldn't find a way to do so, prs are always welcomed in this matter

@noob8boi
Copy link

noob8boi commented Dec 28, 2023 •
edited
Loading

I would like to use the library pytorch_lite for object detection using a YOLO8 model.

However I saw that the size of the APK file is too large, and then I found that the file libpytorch_jni.so is ~74MB. It does not matter whether I specify the version explicitly pytorch_lite: ^4.2.1 or I provide the path: ../ to the pytorch_lite package, the result is roughly the same.

Is there a way to shrink the lib size somehow by removing some unnecessary dependencies?

I was facing the same issue and after some research found out that yolo v8 model can be exported with --optimize flag which will call the _save_for_lite_interpreter function and now the torchscript model will work with lite dependencies. After that I uncommented the dependencies inside the build.gradle file :-

implementation 'org.pytorch:pytorch_android_lite:2.1.0'
implementation 'org.pytorch:pytorch_android_torchvision_lite:2.1.0'

and commented these :-

implementation 'org.pytorch:pytorch_android:2.1.0'
implementation 'org.pytorch:pytorch_android_torchvision:2.1.0'

With that you also have to use LiteModuleLoader.load instead of Module.load for loading your model in PytorchLitePlugin.java file & import org.pytorch.LiteModuleLoader in the same file.

@abdelaziz-mahdy
Copy link
Owner

When I tried that the size didn't change much in my case , can you let me know the difference in size between the normal one and lite one

And if you don't mind can you open a pr?

@noob8boi
Copy link

When I tried that the size didn't change much in my case , can you let me know the difference in size between the normal one and lite one

And if you don't mind can you open a pr?

earlier libpytorch_jni.so was around 70MB but now 18.7MB.. was it ~70MB for you before & after the change?
image

Regarding the PR.. should I make the code changes as mentioned above?

@abdelaziz-mahdy
Copy link
Owner

When I tried that the size didn't change much in my case , can you let me know the difference in size between the normal one and lite one

And if you don't mind can you open a pr?

earlier libpytorch_jni.so was around 70MB but now 18.7MB.. was it ~70MB for you before & after the change?
image

Regarding the PR.. should I make the code changes as mentioned above?

Yes for the pr same changes you mentioned, and I meant the size of the apk using normal and _lite

@noob8boi
Copy link

noob8boi commented Dec 28, 2023 •
edited
Loading

When I tried that the size didn't change much in my case , can you let me know the difference in size between the normal one and lite one
And if you don't mind can you open a pr?

earlier libpytorch_jni.so was around 70MB but now 18.7MB.. was it ~70MB for you before & after the change?
image
Regarding the PR.. should I make the code changes as mentioned above?

Yes for the pr same changes you mentioned, and I meant the size of the apk using normal and _lite

image

i recorded this old build yesterday but for some reason i am unable to reproduce the 330MB build again even after deleting pubspec.lock , clearing all cache etc. Now I keep getting 130MB builds with/without the above changes. Maybe the code change isn't making much of a difference after all 🤔 .. I will reply as soon as I figure this out.

@noob8boi
Copy link

noob8boi commented Dec 28, 2023 •
edited
Loading

Okay so flutter build apk --target-platform android-arm64 --analyze-size outputs two APKs and 18MB is gone with the lite dependency. Still not sure why I was getting 330MB yesterday. Anyway gonna create a PR now 👍

image
image

@noob8boi
Copy link

#68

@abdelaziz-mahdy
Copy link
Owner

Thank you very much, will run workflows and test if everything good

@abdelaziz-mahdy
Copy link
Owner

as i mentioned the difference is not much 5mb
you pr https://github.com/zezo357/pytorch_lite/actions/runs/7348278520/job/20007199904
image

before it
https://github.com/zezo357/pytorch_lite/actions/runs/7185854080/job/19569974871
image

the problem is i dont know if the reduction in size hurts the models options that can be ran? and the reduction is roughly 5mb i dont know if it worth taking the risk

confirm with me if you any other opinion i think it can be published as a work in progress (beta) while we convert to pytorch lite and latest ios version if its stable i think it has some problems #63

@noob8boi
Copy link

noob8boi commented Dec 28, 2023 •
edited
Loading

as i mentioned the difference is not much 5mb you pr https://github.com/zezo357/pytorch_lite/actions/runs/7348278520/job/20007199904 image

before it https://github.com/zezo357/pytorch_lite/actions/runs/7185854080/job/19569974871 image

the problem is i dont know if the reduction in size hurts the models options that can be ran? and the reduction is roughly 5mb i dont know if it worth taking the risk

confirm with me if you any other opinion i think it can be published as a work in progress (beta) while we convert to pytorch lite and latest ios version if its stable i think it has some problems #63

I understand.. actually i posted the comment because i thought this code change will help in reducing libpytorch_jni.so file from 70MB to less than 15MB (libpytorch_jni_lite.so) and solve OP's problem. But I have no clue what was the actual cause of it.

I guess the issue can remain open until further optimizations are found then we can consider creating another PR

@noob8boi
Copy link

if you don't mind can you build output apk file with --analyze-size flag. I want to see the space taken by assets and the libpytorch_jni_lite.so

@abdelaziz-mahdy
Copy link
Owner

if you don't mind can you build output apk file with --analyze-size flag. I want to see the space taken by assets and the libpytorch_jni_lite.so

Sadly I can't do it now, maybe tomorrow and I will let you know

@abdelaziz-mahdy
Copy link
Owner

added it in master and in your pr

@abdelaziz-mahdy
Copy link
Owner

abdelaziz-mahdy commented Dec 28, 2023 •
edited
Loading

pr: https://github.com/zezo357/pytorch_lite/actions/runs/7351145699/job/20014024960

✓  Built build/app/outputs/flutter-apk/app-release.apk (157.1MB).
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
app-release.apk (total compressed)                                        157 MB
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  META-INF/
    CERT.SF                                                                 7 KB
    CERT.RSA                                                              1017 B
    MANIFEST.MF                                                             6 KB
  assets/
    flutter_assets                                                         83 MB
  classes.dex                                                               3 MB
  lib/
    arm64-v8a                                                              15 MB
    armeabi-v7a                                                            21 MB
    Dart AOT symbols accounted decompressed size                            5 MB
      package:flutter                                                       2 MB
      package:image                                                      1023 KB
      dart:core                                                           335 KB
      dart:typed_data                                                     277 KB
      dart:ui                                                             259 KB
      dart:async                                                          108 KB
      dart:collection                                                      97 KB
      dart:convert                                                         71 KB
      dart:ffi                                                             59 KB
      dart:io                                                              54 KB
      package:pytorch_lite_example                                         32 KB
      package:vector_math/
        vector_math_64.dart                                                32 KB
      dart:isolate                                                         31 KB
      package:pytorch_lite                                                 30 KB
      package:camera_platform_interface                                    25 KB
      package:archive                                                      21 KB
      package:path                                                         16 KB
      package:camera_android/
        src                                                                14 KB
      package:camera/
        src                                                                13 KB
      package:image_picker_android                                         10 KB
    x86                                                                    18 MB
    x86_64                                                                 17 MB
  LICENSE-junit.txt                                                         4 KB
  junit/
    runner                                                                  2 KB
  kotlin/
    collections                                                             1 KB
    kotlin.kotlin_builtins                                                  5 KB
    ranges                                                                  1 KB
    reflect                                                                 1 KB
  AndroidManifest.xml                                                       2 KB
  res/
    CG.png                                                                  1 KB
    _W.png                                                                  1 KB
    yq.png                                                                  1 KB
  resources.arsc                                                           [53](https://github.com/zezo357/pytorch_lite/actions/runs/7351145699/job/20014024960#step:6:54) KB
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

master: https://github.com/zezo357/pytorch_lite/actions/runs/7351120618/job/20013953038

Running Gradle task 'assembleRelease'...                          315.4s
✓  Built build/app/outputs/flutter-apk/app-release.apk (175.1MB).
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
app-release.apk (total compressed)                                        175 MB
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  META-INF/
    CERT.SF                                                                 7 KB
    CERT.RSA                                                              1017 B
    MANIFEST.MF                                                             6 KB
  assets/
    flutter_assets                                                         83 MB
  classes.dex                                                               3 MB
  lib/
    arm64-v8a                                                              19 MB
    armeabi-v7a                                                            25 MB
    Dart AOT symbols accounted decompressed size                            5 MB
      package:flutter                                                       2 MB
      package:image                                                      1023 KB
      dart:core                                                           335 KB
      dart:typed_data                                                     277 KB
      dart:ui                                                             259 KB
      dart:async                                                          108 KB
      dart:collection                                                      97 KB
      dart:convert                                                         71 KB
      dart:ffi                                                             59 KB
      dart:io                                                              54 KB
      package:pytorch_lite_example                                         32 KB
      package:vector_math/
        vector_math_64.dart                                                32 KB
      dart:isolate                                                         31 KB
      package:pytorch_lite                                                 30 KB
      package:camera_platform_interface                                    25 KB
      package:archive                                                      21 KB
      package:path                                                         16 KB
      package:camera_android/
        src                                                                14 KB
      package:camera/
        src                                                                13 KB
      package:image_picker_android                                         10 KB
    x86                                                                    22 MB
    x86_64                                                                 22 MB
  LICENSE-junit.txt                                                         4 KB
  junit/
    runner                                                                  2 KB
  kotlin/
    collections                                                             1 KB
    kotlin.kotlin_builtins                                                  5 KB
    ranges                                                                  1 KB
    reflect                                                                 1 KB
  AndroidManifest.xml                                                       2 KB
  res/
    CG.png                                                                  1 KB
    _W.png                                                                  1 KB
    yq.png                                                                  1 KB
  resources.arsc                                                           [53](https://github.com/zezo357/pytorch_lite/actions/runs/7351120618/job/20013953038#step:6:54) KB
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
A summary of your APK analysis can be found at: /Users/runner/.flutter-devtools/apk-code-size-analysis_01.json

@noob8boi
Copy link

noob8boi commented Dec 29, 2023 •
edited
Loading

▒▒▒▒▒▒▒

As you can see 83MB is taken by the flutter assets which I believe is from the model. So I don't think the package can perform any further optimizations over there. And armabi-v7a file is only around 21MB.. so the libpytorch_jni_lite.so is definitely not 70MB .. more like 15MB or so. The analysis json file should be able give in depth view with the help of dart dev tools 🤔.. I hope I'm not drawing incorrect conclusions. Basically I don't think there can be major reduction in sizes now. Best we can do is reduce the model size but that's not something this package is made for.

@abdelaziz-mahdy
Copy link
Owner

▒▒▒▒▒▒▒

As you can see 83MB is taken by the flutter assets which I believe is from the model. So I don't think the package can perform any further optimizations over there. And armabi-v7a file is only around 25MB.. so the libpytorch_jni_lite.so is definitely not 70MB .. more like 15MB or so. The analysis json file should be able give in depth view with the help of dart dev tools 🤔.. I hope I'm not drawing incorrect conclusions. Basically I don't think there can be major reduction in sizes now. Best we can do is reduce the model size but that's not something this package is made for.

Yes I agree the power is in pytorch themselves, for the problem of assets I will try to split app and models so I download models at run time to show true app size

Sadly I can't do it now and don't think I will be able for the next days

Anyway thank you very much for your contribution and for checking the possibility of reducing it to that depth, hopefully pytorch reduces their lib size

@noob8boi
Copy link

▒▒▒▒▒▒▒

As you can see 83MB is taken by the flutter assets which I believe is from the model. So I don't think the package can perform any further optimizations over there. And armabi-v7a file is only around 25MB.. so the libpytorch_jni_lite.so is definitely not 70MB .. more like 15MB or so. The analysis json file should be able give in depth view with the help of dart dev tools 🤔.. I hope I'm not drawing incorrect conclusions. Basically I don't think there can be major reduction in sizes now. Best we can do is reduce the model size but that's not something this package is made for.

Yes I agree the power is in pytorch themselves, for the problem of assets I will try to split app and models so I download models at run time to show true app size

Sadly I can't do it now and don't think I will be able for the next days

Anyway thank you very much for your contribution and for checking the possibility of reducing it to that depth, hopefully pytorch reduces their lib size

happy to help :)

@abdelaziz-mahdy
Copy link
Owner

Btw I will leave your pr open, since it's valid until the assets stuff is handled if it's better then I guess we have a solution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@denesbartha @abdelaziz-mahdy @noob8boi