Wish list: Add the rowStats attributes of the columns of the sys.tasks table

[vogievetsky]

Specifically processed, processedWithError, thrownAway, unparseable

This is currently available from the coordinator task report API but it would be amazing to be able to surface it via DruidSQL

"rowStats":{"buildSegments":{"processed":3630,"processedWithError":0,"thrownAway":0,"unparseable":0}}

Then we could use all the cool SQL magic on it like filtering.

In particular we could surface it in the new Druid console Tasks view as a column:

This will prevent the common confusion of "my task succeed but my where is my data (0 rows were ingested)

[surekhasaharan]

To get rowStats attributes of a task, the overlord api is documented here. In order to get rowStats for each task, we'll have to make a call to this api per task as part of the query. This could potentially slow down retrieval of tasks substantially because it would introduce a network hop for each task. We can speed up the queries by adding a cache, but not sure if it's worth the memory cost, the cache might add.

[stale]

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.

[stale]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.