[dask] Include support for init_score

[jmoralez]

This solves #3807 by including the option to specify init_score as a dask collection to the fit methods of the dask estimators. A test is included to check that using init_score with dask and scikit-learn estimators results in the same result and that not using it results in a different one.

[jameslamb]

Thanks so much for picking this up! Please see my first round of comments.

---

The fit() docs will also need to be updated to include init_score. They currently don't show it, intentionally.

https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.DaskLGBMClassifier.html#lightgbm.DaskLGBMClassifier.fit

Updating the API docs for the Dask estimators is a bit complicated:

• this template is used by both the Dask and sklearn estimators:

  LightGBM/python-package/lightgbm/sklearn.py

  Line 192 in 89da990

  sample_weight : {sample_weight_shape}

  . Note that it contains template strings like {X_shape}
• those template strings are filling in in lines like

  LightGBM/python-package/lightgbm/sklearn.py

  Line 707 in 89da990

  fit.__doc__ = _lgbmmodel_doc_fit.format(

  and

  LightGBM/python-package/lightgbm/dask.py

  Line 907 in 89da990

  _base_doc = _lgbmmodel_doc_fit.format(

• for Dask, we cut out some pieces of the docs in lines like

  LightGBM/python-package/lightgbm/dask.py

  Line 626 in 89da990

  _base_doc = (_base_doc[:_base_doc.find('init_score :')]

  . There are 3 lines like that in dask.py.

If you're interested in updating those, could you try to add init_score to them? You can build and test the docs locally like this:

cd docs/
make clean html
open _build/html/index.html

You can use #3930 as a reference.

If you don't want to mess with the docs that is TOTALLY fine, and I can do it in a follow-up PR.

[jmoralez]

Hi James. I'm having trouble with the tests for this, the results aren't very consistent between runs. Is there something else we can check more easily? I can see that when init_score is set, value for the root node is 0

[jameslamb]

Hi James. I'm having trouble with the tests for this, the results aren't very consistent between runs. Is there something else we can check more easily? I can see that when init_score is set, value for the root node is 0

for now, I think it's fine to just test that dask_model.booster_.trees_to_dataframe()['value'][0] == 0.. As far as I could tell, we don't have any tests on the sklearn module that use init_score, that you could use as a reference. I've been testing some things for the last two days but can't come up with a clean, more effective test. It requires some knowledge of the default objective functions that is a bit beyond my current knowledge.

[jmoralez]

I've included the requested changes, please let me know if you have more comments.

[jameslamb]

This looks great!

Thanks for taking on the documentation as well. I cloned your fork and built the docs to check, and they look good to me.

I'm leaving a "comment" review instead of approving because I'd like to request a review from @StrikerRUS as well before we merge, but from my perspective these changes are approved.

[StrikerRUS]

Many thanks for your contribution!
Just one nit below.

I trust @jameslamb

I cloned your fork and built the docs to check, and they look good to me.

and didn't re-check docs.

[StrikerRUS]

I'd prefer not to mix standard types and numpy ones.

Suggested change

	init_scores = da.full_like(dy, fill_value=init_score, dtype='float')
	init_scores = da.full_like(dy, fill_value=init_score, dtype=np.float64)

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.