LeakyRelu #13488 #13492

kaushal07wick · 2023-03-31T13:12:24Z

No description provided.

kaushal07wick · 2023-03-31T19:05:55Z

Hi @sherry30 , I am trying to implement the LeakyRelu function for the tensorflow raw_ops.
Your, sugestion would be helpful to pass the tests locally.

kaushal07wick · 2023-03-31T19:17:09Z

hey @sherry30 , while running the tests locally, I get this error

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_raw_ops.py:13 (test_tensorflow_LeakyReLU[cpu-ivy.functional.backends.numpy-False-False])
@handle_frontend_test(
>       fn_tree="tensorflow.raw_ops.LeakyRelu",
        dtype_and_x=helpers.dtype_and_values(
            available_dtypes=helpers.get_dtypes("numeric"),
            min_num_dims=1,
        ),
        test_with_out=st.just(False),
    )

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_raw_ops.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
ivy_tests/test_ivy/test_frontends/test_tensorflow/test_raw_ops.py:33: in test_tensorflow_LeakyReLU
    return helpers.test_frontend_function(
ivy_tests/test_ivy/helpers/function_testing.py:897: in test_frontend_function
    raise e
ivy_tests/test_ivy/helpers/function_testing.py:876: in test_frontend_function
    frontend_ret = frontend_fw.__dict__[fn_name](
/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/tf_export.py:400: in wrapper
    return f(**kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_nn_ops.py:5152: in leaky_relu
    _ops.raise_from_not_ok_status(e, name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = _NotOkStatusException(), name = None

    def raise_from_not_ok_status(e, name):
      e.message += (" name: " + name if name is not None else "")
>     raise core._status_to_exception(e) from None  # pylint: disable=protected-access
E     tensorflow.python.framework.errors_impl.InvalidArgumentError: Value for attr 'T' of uint16 is not in the list of allowed values: half, bfloat16, float, double
E     	; NodeDef: {{node LeakyRelu}}; Op<name=LeakyRelu; signature=features:T -> activations:T; attr=alpha:float,default=0.2; attr=T:type,default=DT_FLOAT,allowed=[DT_HALF, DT_BFLOAT16, DT_FLOAT, DT_DOUBLE]> [Op:LeakyRelu]
E     Falsifying example: test_tensorflow_LeakyReLU(
E         dtype_and_x=(['uint16'], [array([0], dtype=uint16)]),
E         fn_tree='ivy.functional.frontends.tensorflow.raw_ops.LeakyRelu',
E         test_flags=num_positional_args=0. with_out=False. inplace=False. native_arrays=[False]. as_variable=[False]. generate_frontend_arrays=False. ,
E         frontend='tensorflow',
E         on_device='cpu',
E     )
E     
E     You can reproduce this example by temporarily adding @reproduce_failure('6.55.0', b'AAQBAAAAAAAAAAAAAA==') as a decorator on your test case

/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py:7164: InvalidArgumentError

sherry30

Hi,

Your PR looks good but a few changes are required.
First off, There are lint errors introduced by your changes, can you fix those?

Other than that, I've requested changes on specific lines.

The error that you're facing seems to be because you're testing on numeric data types while this tf function only expects floats. So you can change this where you're generating data in the test function.

Hope this review was helpful, let me know if you need help with anything else 😃
Thanks

sherry30 · 2023-04-03T09:22:06Z

ivy/functional/frontends/tensorflow/nn.py

@@ -487,3 +487,7 @@ def relu6(features, name=None):
 @to_ivy_arrays_and_back
 def softmax(logits, axis=None, name=None):
    return ivy.softmax(logits, axis=axis)
+
+
+def leaky_relu(features, name=None):


tf.nn.leaky_relu also accepts alpha as an argument, you also have to pass that here

sherry30 · 2023-04-03T09:22:45Z

ivy/functional/frontends/tensorflow/raw_ops.py

+LeakyRelu = to_ivy_arrays_and_back(
+    map_raw_ops_alias(
+        tf_frontend.nn.leaky_relu,
+        kwargs_to_update={"x":"features"},


no need for this line since raw_ops version of leaky_realu has the same argument name

i will just remove this line right

sherry30 · 2023-04-03T09:23:10Z

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_raw_ops.py

@@ -3598,4 +3625,4 @@ def test_tensorflow_BatchMatMulV3(
        Tout=Tout,
        adj_x=adj_x,
        adj_y=adj_y,
-    )
+    )    


accidentally changed... ?

sherry30 · 2023-04-03T09:23:42Z

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_raw_ops.py

+    on_device,
+):
+    dtype, x = dtype_and_x
+    alpha = 0.2  # set the alpha value for LeakyReLU


you shouldn't hard code this value of alpha for the test, you should generate this with hypothesis in the decorator above.

hey @sherry30 , I am getting various error while trying to generate data via hypothesis for the alpha variable.

kindly check on to it. can u just give me a rough outline.

sherry30 · 2023-04-03T09:26:57Z

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_raw_ops.py

+@handle_frontend_test(
+    fn_tree="tensorflow.raw_ops.LeakyRelu",
+    dtype_and_x=helpers.dtype_and_values(
+        available_dtypes=helpers.get_dtypes("numeric"),


change numeric to float since tf.nn.leaky_relu only expects float data types

sherry30

Hi,

Seems like this function hasn't been assigned to you in this ToDo list here. You can follow this doc to get it assigned.

Other than that, Can you remove other extra changes I specified on specific lines.
Also can you add a test for leaky_relu in the nn module since you added that function as well, It will be pretty similar to this one you've added for raw_ops version of leaky_relu.

Let me know if you have any other questions 😄

Thanks

sherry30 · 2023-04-03T19:42:00Z

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_raw_ops.py

@@ -3598,6 +3633,7 @@ def test_tensorflow_BatchMatMulV3(
        Tout=Tout,
        adj_x=adj_x,
        adj_y=adj_y,
+


remove this

sherry30 · 2023-04-03T19:42:08Z

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_raw_ops.py

@@ -3622,3 +3658,4 @@ def test_tensorflow_Size(  # NOQA
        input=x[0],
        out_type=output_dtype,
    )
+


remove this

kaushal07wick · 2023-04-03T19:48:14Z

hey @sherry30 , can u tell me about the data generation for the alpha variable.

kaushal07wick · 2023-04-03T19:52:30Z

Hi,

Seems like this function hasn't been assigned to you in this ToDo list here. You can follow this doc to get it assigned.

Other than that, Can you remove other extra changes I specified on specific lines. Also can you add a test for leaky_relu in the nn module since you added that function as well, It will be pretty similar to this one you've added for raw_ops version of leaky_relu.

Let me know if you have any other questions smile

Thanks

@sherry30 I made the comment for the issue assignment.

kaushal07wick · 2023-04-03T20:09:06Z

@sherry30 I have made some changes, and tried to generate data for alpha but it shows error, should I make a composite strategy for it or would it be unnecessary.

kaushal07wick · 2023-04-03T20:21:53Z

@sherry30 while testing the code, it showed this error.

            _print_traceback_history()
>           raise ivy.utils.exceptions.IvyBackendException(fn.__name__, str(e))
E           ivy.utils.exceptions.IvyBackendException: numpy: leaky_relu: leaky_relu() takes 1 positional argument but 2 were given
E           Falsifying example: test_tensorflow_LeakyReLU(
E               dtype_and_x=(['float64'], [array([-1.])]),
E               alpha=0.5,
E               test_flags=num_positional_args=0. with_out=False. inplace=False. native_arrays=[False]. as_variable=[False]. generate_frontend_arrays=False. ,
E               fn_tree='ivy.functional.frontends.tensorflow.raw_ops.LeakyRelu',
E               frontend='tensorflow',
E               on_device='cpu',
E           )

sherry30

Hi Kaushal,

A composite strategy would be unnecessary since this can be done by simply helpers.floats, let me know if that gives some errors.

The 2nd error you're getting is because ivy.leaky_relu doesn not accept alpha as a positional argument. to fix this I've requested change on specific line.

thanks

sherry30 · 2023-04-04T18:28:55Z

ivy/functional/frontends/tensorflow/nn.py

@@ -489,7 +489,11 @@ def softmax(logits, axis=None, name=None):
    return ivy.softmax(logits, axis=axis)


+def leaky_relu(features, alpha, name=None):
+    return ivy.leaky_relu(features, alpha)


should be alpha=alpha on this line

kaushal07wick · 2023-04-04T18:38:39Z

hi @sherry30 , after making the changes it passed all three tests (numpy, jax, tensorflow) but for the pytorch it failed and showed this error

>           raise ivy.utils.exceptions.IvyBackendException(fn.__name__, str(e))
E           ivy.utils.exceptions.IvyBackendException: torch: leaky_relu: "leaky_relu_cpu" not implemented for 'Half'
E           Falsifying example: test_tensorflow_LeakyReLU(
E               dtype_and_x=(['float16'], [array([-1.], dtype=float16)]),
E               alpha=0.5,
E               test_flags=num_positional_args=0. with_out=False. inplace=False. native_arrays=[False]. as_variable=[False]. generate_frontend_arrays=False. ,
E               fn_tree='ivy.functional.frontends.tensorflow.raw_ops.LeakyRelu',
E               frontend='tensorflow',
E               on_device='cpu',
E           )
E

kaushal07wick · 2023-04-04T18:50:01Z

sherry30

Hi,

Just a few more changes requested. Can you also checkout the lint errors. there are some new introduced from your changes. Once this is done I'll merge it 👍

Thanks

sherry30 · 2023-04-05T13:49:02Z

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_nn.py

@@ -15,6 +15,36 @@
 )


+@handle_frontend_test(
+    fn_tree="tensorflow.raw_ops.LeakyRelu",


this test function is supposed to test the leaky_relu in the nn module.
change it to that function.

sherry30 · 2023-04-05T13:49:28Z

ivy_tests/test_ivy/test_frontends/test_tensorflow/test_nn.py

+    test_with_out=st.just(False),
+    alpha=helpers.floats(min_value=0, max_value=1)
+)
+def test_tensorflow_LeakyReLU(  # NOQA


change the name to test_tensorflow_leaky_relu

ivy/functional/frontends/tensorflow/nn.py

kaushal07wick · 2023-04-05T15:27:52Z

hey @sherry30 , I formatted the code, the lint doesn't shows any error while running locally, but here it shows error.
kindly check once. I don't find any such error.

sherry30 · 2023-04-05T15:47:50Z

You can check the details of the lint error here and then open up Analysing the code with Flake8 and then you can search on the top right bar the file name of the the files you changed and fix them one by one

kaushal07wick · 2023-04-05T15:52:17Z

You can check the details of the lint error here and then open up Analysing the code with Flake8 and then you can search on the top right bar the file name of the the files you changed and fix them one by one

I did that

kaushal07wick · 2023-04-05T18:14:03Z

hey @sherry30 lint errors, from raw_ops and test_raw_ops are removed.
I think you can merge the PR.

sherry30

Alright lgtm 👍

Thanks for the contribution 💪 and sorry for such a long process 😅

kaushal07wick · 2023-04-06T05:38:19Z

Alright lgtm +1

Thanks for the contribution muscle and sorry for such a long process sweat_smile

No worries, thanks @sherry30 for your help.

Co-authored-by: sherry30 <sherrytst30@gmail.com>

kaushal07wick and others added 2 commits March 31, 2023 13:10

leaky_relu_for_tf_frontend

2242625

Merge branch 'unifyai:master' into master

07649d5

ivy-leaves added the TensorFlow Frontend Developing the TensorFlow Frontend, checklist triggered by commenting add_frontend_checklist label Mar 31, 2023

ivy-seed assigned sherry30 Mar 31, 2023

Merge branch 'master' into master

6f97872

sherry30 suggested changes Apr 3, 2023

View reviewed changes

kaushal07wick closed this Apr 3, 2023

kaushal07wick force-pushed the master branch from 6f97872 to d470cde Compare April 3, 2023 18:30

kaushal07wick added 2 commits April 3, 2023 18:32

fixes_for_leakyrelu

0e5a321

fixes

ddbf795

kaushal07wick reopened this Apr 3, 2023

kaushal07wick requested a review from sherry30 April 3, 2023 18:37

hypothesis_test

bc99344

sherry30 suggested changes Apr 3, 2023

View reviewed changes

kaushal07wick requested a review from sherry30 April 3, 2023 19:54

test_func_in_nn_module

0c56672

kaushal07wick added 2 commits April 3, 2023 20:18

some_fixes

6182e9e

alpha_fix

a771655

sherry30 suggested changes Apr 4, 2023

View reviewed changes

kaushal07wick and others added 3 commits April 4, 2023 18:34

Merge branch 'master' into master

5fa14c0

fix_alpha

dbd200e

Merge branch 'master' of github.com:kaushal07wick/ivy

5a3a8b3

kaushal07wick requested a review from sherry30 April 4, 2023 19:24

passed_all_the_tests

a03432f

sherry30 suggested changes Apr 5, 2023

View reviewed changes

kaushal07wick added 3 commits April 5, 2023 14:23

lint_fixes_for_leaky_relu

c3ce9f8

lint_fixes

8fd860d

lint_fix_again

c8ce873

kaushal07wick requested a review from sherry30 April 5, 2023 15:02

Merge branch 'master' into master

eb844f5

kaushal07wick added 3 commits April 5, 2023 18:01

lint_fix

0650df6

fixes

6a0259e

leaky_relu_for_tf_frontend

6e24a46

sherry30 approved these changes Apr 6, 2023

View reviewed changes

sherry30 merged commit 946d3e3 into ivy-llc:master Apr 6, 2023

kaushal07wick mentioned this pull request Apr 6, 2023

LeakyRelu #13488

Closed

MuhammedAshraf2020 pushed a commit to MuhammedAshraf2020/ivy that referenced this pull request Apr 9, 2023

LeakyRelu ivy-llc#13488 (ivy-llc#13492)

2078411

Co-authored-by: sherry30 <sherrytst30@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LeakyRelu #13488 #13492

LeakyRelu #13488 #13492

kaushal07wick commented Mar 31, 2023

kaushal07wick commented Mar 31, 2023

kaushal07wick commented Mar 31, 2023

sherry30 left a comment •

edited

Loading

sherry30 Apr 3, 2023

sherry30 Apr 3, 2023

kaushal07wick Apr 3, 2023

sherry30 Apr 3, 2023

kaushal07wick Apr 3, 2023

sherry30 Apr 3, 2023

kaushal07wick Apr 3, 2023

sherry30 Apr 3, 2023

sherry30 left a comment

sherry30 Apr 3, 2023

sherry30 Apr 3, 2023

kaushal07wick commented Apr 3, 2023

kaushal07wick commented Apr 3, 2023

kaushal07wick commented Apr 3, 2023

kaushal07wick commented Apr 3, 2023

sherry30 left a comment

sherry30 Apr 4, 2023

kaushal07wick commented Apr 4, 2023

kaushal07wick commented Apr 4, 2023

sherry30 left a comment

sherry30 Apr 5, 2023

kaushal07wick Apr 5, 2023

sherry30 Apr 5, 2023

kaushal07wick Apr 5, 2023

kaushal07wick commented Apr 5, 2023

sherry30 commented Apr 5, 2023 •

edited

Loading

kaushal07wick commented Apr 5, 2023

kaushal07wick commented Apr 5, 2023

sherry30 left a comment

kaushal07wick commented Apr 6, 2023

LeakyRelu #13488 #13492

LeakyRelu #13488 #13492

Conversation

kaushal07wick commented Mar 31, 2023

kaushal07wick commented Mar 31, 2023

kaushal07wick commented Mar 31, 2023

sherry30 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sherry30 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaushal07wick commented Apr 3, 2023

kaushal07wick commented Apr 3, 2023

kaushal07wick commented Apr 3, 2023

kaushal07wick commented Apr 3, 2023

sherry30 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaushal07wick commented Apr 4, 2023

kaushal07wick commented Apr 4, 2023

sherry30 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaushal07wick commented Apr 5, 2023

sherry30 commented Apr 5, 2023 • edited Loading

kaushal07wick commented Apr 5, 2023

kaushal07wick commented Apr 5, 2023

sherry30 left a comment

Choose a reason for hiding this comment

kaushal07wick commented Apr 6, 2023

sherry30 left a comment •

edited

Loading

sherry30 commented Apr 5, 2023 •

edited

Loading