Handling multiple rate-limits

[subnetmarco]

Closes issue #205 and deprecates #224.

This pull requests enables support for multiple rate-limit periods on the same API.

New plugin configuration value format

This PR promotes a new format for setting up the ratelimiting plugin, whose value is now value.{period}={limit}, for example:

value.second=10 instead of the previous one value.limit=10&value.period=second.

The plugin will transparently handle the old value format without requiring a migration of the data.

[thibaultcha]

Sounds good 👍. Except for the manual on_insert trigger. It became self_check and is recursively called on sub-schemas now. Also called during insert and update. It's not yet merged in master.

[thibaultcha]

However I am not sure about supporting the old format. Because now we are either forever supporting a deprecated format, or just fighting off an inevitable deprecation.

We could drop it now, while we are still not in 1.0, and optionally provide a migration script and instructions on how to run it. Instead of dropping it later on when old providers will have both formats in their plugins configuration (the old and the new one) and it will be harder to migrate.

[subnetmarco]

We could drop it now, while we are still not in 1.0, and optionally provide a migration script and instructions on how to run it. Instead of dropping it later on when old providers will have both formats in their plugins configuration (the old and the new one) and it will be harder to migrate.

I thought of it, and I was about to implement it. Then I thought more carefully and I found the following reasons to postpone it:

• Database schema migrations have never been thought in this scenario where data migration is involved too (till now we did schema migrations), and giving that 0.4.0 will be released soon we might not have enough time to think carefully on the best implementation.
• A data migration script will decrease adoption and create problems if not done correctly, and rate-limiting is one of the most used plugins. So in order to avoid potential problems for 0.4.0, or decrease adoption because migration is involved, I believe it's better to postpone it.
• In this specific case retro-compatibility is very easy to maintain (it's an if..else statement)
• New migrations might be necessary, some of them might be more deep. Because we don't really know what to expect from the future from a schema/data migration perspective, we could decide to keep retro-compatibility now (as long as it's so easy to implement) and break it when switching to 1.0 (basically no migrations now, but when adopting 1.0 a full-scale migration would be required because retro-compatibility to pre-1.0 versions would not be maintained).
• We also talked about merging all the migration script files into one file for 1.0, so schema retro-compatibility will be broken anyways if we decided to merge these files and a full-scale migration will be required anyways for 1.0

Thoughts?

[subnetmarco]

@thibaultcha to fix the problem I had to implement this check. Basically this is the problem:

• BaseDao invokes the validate_entity. The entity is the plugin_configuration, which has a self_check.
• The self_check is never being invoked on the underlying plugin's schema.
• This code invokes the underlying plugins's self_check if it exists.

[thibaultcha]

Sounds good, but you are locking the self_check to a SCHEMA error, and it shouldn't be. It can be any other error type too. You should return the error as it is returned by self_check, it is up to the plugin to decide what error it is exactly. Maybe the error comes from Cassandra being unaccessible for example.

Otherwise the check sounds good to me. A better solution would be to investigate on why it is not invoked from validate_entity, because the function calls itself on sub-schemas so if there is no other error, self_check should be recursively called already.

[subnetmarco]

because the function calls itself on sub-schemas

I believe this part is missing.

[tamizhgeek]

Sorry to comment on a closed PR, but do we have the requested benchmarks when a API is configured with multiple rate limits? Just curious!