Format is being misapplied

[alexanderdean]

I'm seeing lots of strings getting this pattern applied:

"^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{4}|[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)$"

What does this signify? I am pretty sure that the properties in question are strings but shouldn't match the above format...

        },
        "author": {
          "type": "string",
          "pattern": "^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{4}|[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)$"
        },
        "breadcrumb": {
          "type": "array",
          "items": {
            "type": "string",
            "pattern": "^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{4}|[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)$"
          }
        },

[chuwy]

This is base64 pattern, as dicussed in #58

[alexanderdean]

Hmm, interesting - I guess the trouble is that lots of short strings can be mis-identified as base64...

[chuwy]

I encountered it only once or twice. But this will be of course eliminated on any decent set of JSONs.
Also we can add length restriction, as it's not very common thing to encode short strings. Better if no pattern will be added at all.

[alexanderdean]

Length restriction is a good idea - once a field is above a certain length, if it has no spaces or punctuation then yes highly likely to be base64...

[chuwy]

Ok, then we need to decide exact length. 32?

[alexanderdean]

Let's discuss before work is started on 0.3.0

[chuwy]

I just got an idea for another option. With scalaz refactoring reduce operation will be moved out of merge and thus we can apply different reduce depending on how much instances was processed.
For example, if we processing 1-2 instances, this pattern won't be used at all. If this pattern encountered on >10 than let it be base64.

[alexanderdean]

Good idea, so let's push back to 0.4.0

[alexanderdean]

This didn't auto-close. Is it done in 0.4.0 @chuwy ?

[chuwy]

Yes, it was done. I don't know why it didn't get closed.

[alexanderdean]

No worries, closing.