Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collector skips metrics that have no sources, but do have one or more issues #3221

Closed
fniessink opened this issue Jan 10, 2022 · 1 comment · Fixed by #3235
Closed

Collector skips metrics that have no sources, but do have one or more issues #3221

fniessink opened this issue Jan 10, 2022 · 1 comment · Fixed by #3235
Assignees
Labels
Bug Something isn't working

Comments

@fniessink
Copy link
Member

fniessink commented Jan 10, 2022

This method checks whether the user has specified all mandatory parameters for all sources. Even if the metric has one or more issues, if not all mandatory parameters are present, the metric is skipped. If there are no sources at all, the metric is also skipped.

    def __can_collect(self, metric: dict) -> bool:
        """Return whether the user has specified all mandatory parameters for all sources."""
        sources = metric.get("sources", {})
        for source in sources.values():
            parameters = self.data_model.get("sources", {}).get(source["type"], {}).get("parameters", {})
            for parameter_key, parameter in parameters.items():
                if (
                    parameter.get("mandatory")
                    and metric["type"] in parameter.get("metrics")
                    and not source.get("parameters", {}).get(parameter_key)
                    and not parameter.get("default_value")
                ):
                    return False
        return bool(sources)
@fniessink fniessink added the Bug Something isn't working label Jan 10, 2022
@fniessink
Copy link
Member Author

fniessink commented Jan 12, 2022

Move the parameter check to SourceCollector.collect(), and if it fails, skip collecting and parsing the source responses.

    async def collect(self) -> SourceMeasurement:
        """Return the measurement from this source."""
        if self.__all_mandatory_parameters_ok(...):
            responses = await self.__safely_get_source_responses()
            measurement = await self.__safely_parse_source_responses(responses)
            measurement.api_url = responses.api_url
            measurement.landing_url = await self.__safely_parse_landing_url(responses)
        else:
            measurement = SourceMeasurement()
        return measurement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment