Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add separate instrumentation for redis.asyncio.client #807

Closed
ahmedhr opened this issue May 2, 2023 · 3 comments
Closed

Add separate instrumentation for redis.asyncio.client #807

ahmedhr opened this issue May 2, 2023 · 3 comments

Comments

@ahmedhr
Copy link
Contributor

ahmedhr commented May 2, 2023

Reduce overhead of aioredis version when using redis.asyncio

Description
Evaluating the existing aioredis version raises a few exception when aioredis is not installed and since you are not using aioredis this check is invalid now.

Expected Behavior
For redis.asyncio.client, we need to separate out the instrumentation function, currently the aioredis intrumentation is used

Steps to Reproduce
we should have a separate instrument_aioredis_client and _wrap_AioRedis_method_wrapper function for redis.asyncio.client. And remove aioredis_version = get_package_version_tuple("aioredis").

Since get_package_version_tuple raises multiple exceptions and finally go to

            else:
                from redis.asyncio.client import Pipeline

when we execute multiple commands, the overhead is too much, check the below profiling.

import newrelic.agent
import redis.asyncio as redis
import asyncio
import pstats
import time

from redis.asyncio.connection import ConnectionPool

pool = ConnectionPool.from_url('redis://127.0.0.1:6379/?decode_responses=true')

redis_conn = redis.Redis(connection_pool=pool)
newrelic.agent.initialize()

async def profile__nr_wrapper_AioRedis_method_():
    pr = cProfile.Profile()
    pr.enable()

    start_time = time.time()
    for _ in range(1000):
        await redis_conn.get("Test one")

    print(f"Time taken: {time.time() - start_time}")
    pr.disable()
    ps = pstats.Stats(pr).sort_stats("cumulative")
    ps.strip_dirs().print_stats("profile__nr_wrapper_AioRedis_method_")


asyncio.run(profile__nr_wrapper_AioRedis_method_())

Output:

Time taken: 1.4444618225097656
         4708172 function calls (4707770 primitive calls) in 1.438 seconds

   Random listing order was used
   List reduced from 438 to 1 due to restriction <'profile__nr_wrapper_AioRedis_method_'>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      202    0.000    0.000    1.343    0.007 kindex_radius.py:128(profile__nr_wrapper_AioRedis_method_)

Total time taken is 1.44 and 1.343 is of nr_wrapper_AioRedis_method

Also get_package_version_tuple is the main culprit of the issue.

and without newrelic.agent.initialize() or installing aioredis*(prevents redis.async.client function to run - A different bug that'll also be solved)

Output without newrelic.agent.initialize() :

Time taken: 0.054537057876586914
         66135 function calls in 0.054 seconds

   Random listing order was used
   List reduced from 218 to 1 due to restriction <'profile__nr_wrapper_AioRedis_method_'>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      202    0.000    0.000    0.003    0.000 kindex_radius.py:128(profile__nr_wrapper_AioRedis_method_)
@stale
Copy link

stale bot commented Aug 12, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Aug 12, 2023
@lrafeei
Copy link
Contributor

lrafeei commented Aug 14, 2023

I will close this ticket--Redis.asyncio.client is now supported and there is work underway for improving the performance for get_package_version/get_package_version_tuple, which will be released alongside Python 3.12 support (within the next two months)

@lrafeei lrafeei closed this as completed Aug 14, 2023
@ahmedhr
Copy link
Contributor Author

ahmedhr commented Aug 14, 2023

Thanks @lrafeei.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants