Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ML module #62

Open
abdelaziz-mahdy opened this issue May 23, 2024 · 13 comments · May be fixed by #63
Open

Add ML module #62

abdelaziz-mahdy opened this issue May 23, 2024 · 13 comments · May be fixed by #63
Labels
enhancement New feature or request question Further information is requested

Comments

@abdelaziz-mahdy
Copy link
Contributor

Question

i know that ML module is not supported yet, any way i could help with that since i want to use CascadeClassifier i think its in the ml module (i didnt check )

if i cant help with it, any time plan for it?

@abdelaziz-mahdy abdelaziz-mahdy added the question Further information is requested label May 23, 2024
@rainyl
Copy link
Owner

rainyl commented May 23, 2024

@abdelaziz-mahdy

If you just need CascadeClassifier, it is included in objdetect module, and has been implemented, https://docs.opencv.org/4.x/d1/de5/classcv_1_1CascadeClassifier.html

class CascadeClassifier extends CvStruct<cvg.CascadeClassifier> {

test('cv.CascadeClassifier', () {

It seems training machine learning models using python and other tools is a more common and fashion way nowadays, and I have no much free time, so ML module was not implemented and not planned recently, maybe it will be added several months later when I have some time.

Anyway, it is true that some people need ML module, I(and other users) will be grateful if you can implement it.

@abdelaziz-mahdy
Copy link
Contributor Author

Thank you very much for the links and the awesome package, I will check what needs to be implemented and try to do it

@rainyl
Copy link
Owner

rainyl commented May 23, 2024

😄

@rainyl rainyl changed the title ML module plans Add ML module May 23, 2024
@rainyl rainyl added the enhancement New feature or request label May 23, 2024
@abdelaziz-mahdy
Copy link
Contributor Author

abdelaziz-mahdy commented May 23, 2024

i am trying to make some guide lines so that i follow if its okay

  1. Branch Selection:

    • Should I work on the main branch or the native assets branch? since The native assets branch has many changes.
  2. Workflow with ffigen:

    • Add the desired implementation in the .h files.
    • Implement the logic in the .cpp files.
    • Add the interface in Dart.
    • Create the test cases for the Dart interface.
  3. Checklist:

    • Determine the correct branch to work on. main
    • Add desired implementation in .h files.
    • Implement logic in .cpp files.
    • Run ffigen to generate bindings
    • Add interface in Dart.
    • Create test cases for the Dart interface.

Questions:

  1. OpenCV on iOS:
    • Have you tried using OpenCV on a clean project in iOS?
    • I remember having an issue with iOS and FFI where the C++ files get ignored, and the developer has to import them manually. I will check if this is still a problem. The native assets should fix this issue.

note: this is my package where i had the problem https://github.com/abdelaziz-mahdy/pytorch_lite/blob/latest-ffi/ios/pytorch_lite.podspec but i see that you created .framework which should not have this problem so ignore this question,

@rainyl
Copy link
Owner

rainyl commented May 23, 2024

wow, you are the author of pytorch_lite, cool 👍

For the questions:

  1. Branch Selection: develop on a new branch based on main, it is easy to update native-assets once finished on main.
  2. OpenCV on iOS: as you said, for ios the .framework is used, I have no apple devices so not tested, but other developers tested and it worked.

I have edited your checklist, hope you won't mind it.

BTW, you can find more build instructions in workflow file, thanks for your efforts in advance.

@rainyl
Copy link
Owner

rainyl commented May 23, 2024

One more thing, to speedup the build of opencv_dart, opencv itself is built in another repo with ML disabled, I have enabled it but the workflow has not finished yet, generally it will take about 40 min, once finished, the published tag will be 4.9.0+3 so you need to change the opencv version in conanfile.py to "4.9.0+3" at

OPENCV_VERSION = "4.9.0+2"

You can check the build process here: OPENCV build release

@abdelaziz-mahdy
Copy link
Contributor Author

abdelaziz-mahdy commented May 23, 2024

Oh thank you for that, is there is a guide you follow of what the headers are and the implementation for the functions? I checked gocv but looks like there is no ml module or I can't find it,

If there is something I can take a reference that will make it much easier

edit: also i see that the build finished but the release doesnt have the files https://github.com/rainyl/opencv.full/releases/tag/4.9.0%2B3

@rainyl
Copy link
Owner

rainyl commented May 23, 2024

Yes, this project was originally developed based on gocv, but now nearly every API has been refactored to cache exceptions, the most important guide is the documentation of opencv, i.e., wrap the classes and functions with C-style functions.

You can take a look at cv.Subdiv2D in this PR: #60

headers:

CvStatus Subdiv2D_NewEmpty(Subdiv2D *rval);
CvStatus Subdiv2D_NewWithRect(Rect rect, Subdiv2D *rval);
void Subdiv2D_Close(Subdiv2D *self);
CvStatus Subdiv2D_EdgeDst(Subdiv2D self, int edge, Point2f *dstpt, int *rval);
CvStatus Subdiv2D_EdgeOrg(Subdiv2D self, int edge, Point2f *orgpt, int *rval);
CvStatus Subdiv2D_FindNearest(Subdiv2D self, Point2f pt, Point2f *nearestPt, int *rval);
CvStatus Subdiv2D_GetEdge(Subdiv2D self, int edge, int nextEdgeType, int *rval);
CvStatus Subdiv2D_GetEdgeList(Subdiv2D self, Vec4f **rval, int *size);
CvStatus Subdiv2D_GetLeadingEdgeList(Subdiv2D self, VecInt *leadingEdgeList);
CvStatus Subdiv2D_GetTriangleList(Subdiv2D self, Vec6f **rval, int *size);
CvStatus Subdiv2D_GetVertex(Subdiv2D self, int vertex, int *firstEdge, Point2f *rval);
CvStatus Subdiv2D_GetVoronoiFacetList(Subdiv2D self, VecInt idx, VecVecPoint2f *facetList,
VecPoint2f *facetCenters);
CvStatus Subdiv2D_InitDelaunay(Subdiv2D self, Rect rect);
CvStatus Subdiv2D_Insert(Subdiv2D self, Point2f pt, int *rval);
CvStatus Subdiv2D_InsertVec(Subdiv2D self, VecPoint2f ptvec);
CvStatus Subdiv2D_Locate(Subdiv2D self, Point2f pt, int *edge, int *vertex, int *rval);
CvStatus Subdiv2D_NextEdge(Subdiv2D self, int edge, int *rval);
CvStatus Subdiv2D_RotateEdge(Subdiv2D self, int edge, int rotate, int *rval);
CvStatus Subdiv2D_SymEdge(Subdiv2D self, int edge, int *rval);

implementation:

CvStatus Subdiv2D_NewEmpty(Subdiv2D *rval)
{
BEGIN_WRAP
*rval = {new cv::Subdiv2D()};
END_WRAP
}
CvStatus Subdiv2D_NewWithRect(Rect rect, Subdiv2D *rval)
{
BEGIN_WRAP
*rval = {new cv::Subdiv2D(cv::Rect(rect.x, rect.y, rect.width, rect.height))};
END_WRAP
}
void Subdiv2D_Close(Subdiv2D *self){CVD_FREE(self)}
CvStatus Subdiv2D_EdgeDst(Subdiv2D self, int edge, Point2f *dstpt, int *rval)
{
BEGIN_WRAP
auto p = cv::Point2f();
*rval = self.ptr->edgeDst(edge, &p);
*dstpt = {p.x, p.y};
END_WRAP
}
CvStatus Subdiv2D_EdgeOrg(Subdiv2D self, int edge, Point2f *orgpt, int *rval)
{
BEGIN_WRAP
auto p = cv::Point2f();
*rval = self.ptr->edgeOrg(edge, &p);
*orgpt = {p.x, p.y};
END_WRAP
}
CvStatus Subdiv2D_FindNearest(Subdiv2D self, Point2f pt, Point2f *nearestPt, int *rval)
{
BEGIN_WRAP
auto p = cv::Point2f();
*rval = self.ptr->findNearest(cv::Point2f(pt.x, pt.y), &p);
*nearestPt = {p.x, p.y};
END_WRAP
}
CvStatus Subdiv2D_GetEdge(Subdiv2D self, int edge, int nextEdgeType, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->getEdge(edge, nextEdgeType);
END_WRAP
}
CvStatus Subdiv2D_GetEdgeList(Subdiv2D self, Vec4f **rval, int *size)
{
BEGIN_WRAP
auto v = std::vector<cv::Vec4f>();
self.ptr->getEdgeList(v);
*size = v.size();
auto rv = new Vec4f[v.size()];
for (int i = 0; i < v.size(); i++) {
rv[i] = {v[i].val[0], v[i].val[1], v[i].val[2], v[i].val[3]};
}
*rval = rv;
END_WRAP
}
CvStatus Subdiv2D_GetLeadingEdgeList(Subdiv2D self, VecInt *leadingEdgeList)
{
BEGIN_WRAP
auto v = new std::vector<int>();
self.ptr->getLeadingEdgeList(*v);
*leadingEdgeList = {v};
END_WRAP
}
CvStatus Subdiv2D_GetTriangleList(Subdiv2D self, Vec6f **rval, int *size)
{
BEGIN_WRAP
auto v = std::vector<cv::Vec6f>();
self.ptr->getTriangleList(v);
*size = v.size();
auto rv = new Vec6f[v.size()];
for (int i = 0; i < v.size(); i++) {
rv[i] = {v[i].val[0], v[i].val[1], v[i].val[2], v[i].val[3], v[i].val[4], v[i].val[5]};
}
*rval = rv;
END_WRAP
}
CvStatus Subdiv2D_GetVertex(Subdiv2D self, int vertex, int *firstEdge, Point2f *rval)
{
BEGIN_WRAP
auto p = self.ptr->getVertex(vertex, firstEdge);
*rval = {p.x, p.y};
END_WRAP
}
CvStatus Subdiv2D_GetVoronoiFacetList(Subdiv2D self, VecInt idx, VecVecPoint2f *facetList,
VecPoint2f *facetCenters)
{
BEGIN_WRAP
auto vf = std::vector<std::vector<cv::Point2f>>();
auto vfc = std::vector<cv::Point2f>();
self.ptr->getVoronoiFacetList(*idx.ptr, vf, vfc);
*facetList = {new std::vector<std::vector<cv::Point2f>>(vf)};
*facetCenters = {new std::vector<cv::Point2f>(vfc)};
END_WRAP;
}
CvStatus Subdiv2D_InitDelaunay(Subdiv2D self, Rect rect)
{
BEGIN_WRAP
self.ptr->initDelaunay(cv::Rect(rect.x, rect.y, rect.width, rect.height));
END_WRAP
}
CvStatus Subdiv2D_Insert(Subdiv2D self, Point2f pt, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->insert(cv::Point2f(pt.x, pt.y));
END_WRAP
}
CvStatus Subdiv2D_InsertVec(Subdiv2D self, VecPoint2f ptvec)
{
BEGIN_WRAP
self.ptr->insert(*ptvec.ptr);
END_WRAP
}
CvStatus Subdiv2D_Locate(Subdiv2D self, Point2f pt, int *edge, int *vertex, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->locate(cv::Point2f(pt.x, pt.y), *edge, *vertex);
END_WRAP
}
CvStatus Subdiv2D_NextEdge(Subdiv2D self, int edge, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->nextEdge(edge);
END_WRAP
}
CvStatus Subdiv2D_RotateEdge(Subdiv2D self, int edge, int rotate, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->rotateEdge(edge, rotate);
END_WRAP
}
CvStatus Subdiv2D_SymEdge(Subdiv2D self, int edge, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->symEdge(edge);
END_WRAP
}

Edit: Now published

@abdelaziz-mahdy
Copy link
Contributor Author

Yes, this project was originally developed based on gocv, but now nearly every API has been refactored to cache exceptions, the most important guide is the documentation of opencv, i.e., wrap the classes and functions with C-style functions.

You can take a look at cv.Subdiv2D in this PR: #60

headers:

CvStatus Subdiv2D_NewEmpty(Subdiv2D *rval);
CvStatus Subdiv2D_NewWithRect(Rect rect, Subdiv2D *rval);
void Subdiv2D_Close(Subdiv2D *self);
CvStatus Subdiv2D_EdgeDst(Subdiv2D self, int edge, Point2f *dstpt, int *rval);
CvStatus Subdiv2D_EdgeOrg(Subdiv2D self, int edge, Point2f *orgpt, int *rval);
CvStatus Subdiv2D_FindNearest(Subdiv2D self, Point2f pt, Point2f *nearestPt, int *rval);
CvStatus Subdiv2D_GetEdge(Subdiv2D self, int edge, int nextEdgeType, int *rval);
CvStatus Subdiv2D_GetEdgeList(Subdiv2D self, Vec4f **rval, int *size);
CvStatus Subdiv2D_GetLeadingEdgeList(Subdiv2D self, VecInt *leadingEdgeList);
CvStatus Subdiv2D_GetTriangleList(Subdiv2D self, Vec6f **rval, int *size);
CvStatus Subdiv2D_GetVertex(Subdiv2D self, int vertex, int *firstEdge, Point2f *rval);
CvStatus Subdiv2D_GetVoronoiFacetList(Subdiv2D self, VecInt idx, VecVecPoint2f *facetList,
VecPoint2f *facetCenters);
CvStatus Subdiv2D_InitDelaunay(Subdiv2D self, Rect rect);
CvStatus Subdiv2D_Insert(Subdiv2D self, Point2f pt, int *rval);
CvStatus Subdiv2D_InsertVec(Subdiv2D self, VecPoint2f ptvec);
CvStatus Subdiv2D_Locate(Subdiv2D self, Point2f pt, int *edge, int *vertex, int *rval);
CvStatus Subdiv2D_NextEdge(Subdiv2D self, int edge, int *rval);
CvStatus Subdiv2D_RotateEdge(Subdiv2D self, int edge, int rotate, int *rval);
CvStatus Subdiv2D_SymEdge(Subdiv2D self, int edge, int *rval);

implementation:

CvStatus Subdiv2D_NewEmpty(Subdiv2D *rval)
{
BEGIN_WRAP
*rval = {new cv::Subdiv2D()};
END_WRAP
}
CvStatus Subdiv2D_NewWithRect(Rect rect, Subdiv2D *rval)
{
BEGIN_WRAP
*rval = {new cv::Subdiv2D(cv::Rect(rect.x, rect.y, rect.width, rect.height))};
END_WRAP
}
void Subdiv2D_Close(Subdiv2D *self){CVD_FREE(self)}
CvStatus Subdiv2D_EdgeDst(Subdiv2D self, int edge, Point2f *dstpt, int *rval)
{
BEGIN_WRAP
auto p = cv::Point2f();
*rval = self.ptr->edgeDst(edge, &p);
*dstpt = {p.x, p.y};
END_WRAP
}
CvStatus Subdiv2D_EdgeOrg(Subdiv2D self, int edge, Point2f *orgpt, int *rval)
{
BEGIN_WRAP
auto p = cv::Point2f();
*rval = self.ptr->edgeOrg(edge, &p);
*orgpt = {p.x, p.y};
END_WRAP
}
CvStatus Subdiv2D_FindNearest(Subdiv2D self, Point2f pt, Point2f *nearestPt, int *rval)
{
BEGIN_WRAP
auto p = cv::Point2f();
*rval = self.ptr->findNearest(cv::Point2f(pt.x, pt.y), &p);
*nearestPt = {p.x, p.y};
END_WRAP
}
CvStatus Subdiv2D_GetEdge(Subdiv2D self, int edge, int nextEdgeType, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->getEdge(edge, nextEdgeType);
END_WRAP
}
CvStatus Subdiv2D_GetEdgeList(Subdiv2D self, Vec4f **rval, int *size)
{
BEGIN_WRAP
auto v = std::vector<cv::Vec4f>();
self.ptr->getEdgeList(v);
*size = v.size();
auto rv = new Vec4f[v.size()];
for (int i = 0; i < v.size(); i++) {
rv[i] = {v[i].val[0], v[i].val[1], v[i].val[2], v[i].val[3]};
}
*rval = rv;
END_WRAP
}
CvStatus Subdiv2D_GetLeadingEdgeList(Subdiv2D self, VecInt *leadingEdgeList)
{
BEGIN_WRAP
auto v = new std::vector<int>();
self.ptr->getLeadingEdgeList(*v);
*leadingEdgeList = {v};
END_WRAP
}
CvStatus Subdiv2D_GetTriangleList(Subdiv2D self, Vec6f **rval, int *size)
{
BEGIN_WRAP
auto v = std::vector<cv::Vec6f>();
self.ptr->getTriangleList(v);
*size = v.size();
auto rv = new Vec6f[v.size()];
for (int i = 0; i < v.size(); i++) {
rv[i] = {v[i].val[0], v[i].val[1], v[i].val[2], v[i].val[3], v[i].val[4], v[i].val[5]};
}
*rval = rv;
END_WRAP
}
CvStatus Subdiv2D_GetVertex(Subdiv2D self, int vertex, int *firstEdge, Point2f *rval)
{
BEGIN_WRAP
auto p = self.ptr->getVertex(vertex, firstEdge);
*rval = {p.x, p.y};
END_WRAP
}
CvStatus Subdiv2D_GetVoronoiFacetList(Subdiv2D self, VecInt idx, VecVecPoint2f *facetList,
VecPoint2f *facetCenters)
{
BEGIN_WRAP
auto vf = std::vector<std::vector<cv::Point2f>>();
auto vfc = std::vector<cv::Point2f>();
self.ptr->getVoronoiFacetList(*idx.ptr, vf, vfc);
*facetList = {new std::vector<std::vector<cv::Point2f>>(vf)};
*facetCenters = {new std::vector<cv::Point2f>(vfc)};
END_WRAP;
}
CvStatus Subdiv2D_InitDelaunay(Subdiv2D self, Rect rect)
{
BEGIN_WRAP
self.ptr->initDelaunay(cv::Rect(rect.x, rect.y, rect.width, rect.height));
END_WRAP
}
CvStatus Subdiv2D_Insert(Subdiv2D self, Point2f pt, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->insert(cv::Point2f(pt.x, pt.y));
END_WRAP
}
CvStatus Subdiv2D_InsertVec(Subdiv2D self, VecPoint2f ptvec)
{
BEGIN_WRAP
self.ptr->insert(*ptvec.ptr);
END_WRAP
}
CvStatus Subdiv2D_Locate(Subdiv2D self, Point2f pt, int *edge, int *vertex, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->locate(cv::Point2f(pt.x, pt.y), *edge, *vertex);
END_WRAP
}
CvStatus Subdiv2D_NextEdge(Subdiv2D self, int edge, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->nextEdge(edge);
END_WRAP
}
CvStatus Subdiv2D_RotateEdge(Subdiv2D self, int edge, int rotate, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->rotateEdge(edge, rotate);
END_WRAP
}
CvStatus Subdiv2D_SymEdge(Subdiv2D self, int edge, int *rval)
{
BEGIN_WRAP
*rval = self.ptr->symEdge(edge);
END_WRAP
}

nice i like how clean this is done, i will check what i can do tomorrow and open a pr if i was able to implement it.

thank again for your help, this info will hopefully make it much easier for me to make the pr <3

@abdelaziz-mahdy
Copy link
Contributor Author

abdelaziz-mahdy commented May 23, 2024

ffigen fails in files not related to the file i made
image
is that a problem from my side?

running conan build . -b missing compiled successfully

adding
#include <stddef.h> before core file made it work, dont know if that the correct fix

@rainyl
Copy link
Owner

rainyl commented May 23, 2024

Well, I developed on windows and it worked normally, maybe macos needs stddef.h, feel free to add it.

@rainyl
Copy link
Owner

rainyl commented May 23, 2024

I have published a new branch ml with some data structures wrapped, but they were not tested, just for some detailed instructions and methods to provide some help, but forgive me that I have no more time to finish it, take a look and hoping it can help you~

@abdelaziz-mahdy
Copy link
Contributor Author

Nice, thank you it helps, also now I know that you would prefer if each class has separate files, since I was implementing them in one file

And the cmake change, I forgot that part

Anyway I will try to implement it and let's see if I achieve something good ❤️

@rainyl rainyl linked a pull request May 25, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants