Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sanitization reporting #912

Merged
merged 43 commits into from
Feb 9, 2018
Merged

Add sanitization reporting #912

merged 43 commits into from
Feb 9, 2018

Commits on Jan 27, 2018

  1. Configuration menu
    Copy the full SHA
    0a44284 View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2018

  1. Issue #843: Tracking for removed nodes and attributes.

    Building upon Weston's work and solution design,
    Add a class to track whenever a node or attribute is removed.
    And a method to get whether a node was removed.
    The format of the stored nodes and attributes might change.
    This will probably depend on the error reporting needed
    in the REST API and GET request response.
    Ryan Kienstra committed Jan 28, 2018
    Configuration menu
    Copy the full SHA
    e9f394a View commit details
    Browse the repository at this point in the history
  2. Issue #843: Correct a failed Travis build by excluding a PHPCS rule.

    There was an error:
    Class file names should be based on the class name with 'class-'
    But the format of the other test files is different.
    So use that format, and exclude this rule for test files.
    Ryan Kienstra committed Jan 28, 2018
    Configuration menu
    Copy the full SHA
    d7104b8 View commit details
    Browse the repository at this point in the history
  3. Issue #843: Add a method to process markup for AMP validtity.

    The 'mutation_callback' will then track removed nodes and attributes.
    Also, change the way in which we pass the 'mutation_callback.'
    Before, it was part of the constructor of:
    AMP_Tag_And_Attribute_Sanitizer.
    Instead, move it to the $args of:
    AMP_Content_Sanitizer::sanitize().
    This will pass it to all of the sanitizer/* files when they're instantiated.
    @todo: look at whether to call the callback for all node removals.
    Ryan Kienstra committed Jan 28, 2018
    Configuration menu
    Copy the full SHA
    616262f View commit details
    Browse the repository at this point in the history
  4. Issue #843: Track removed iframes in a helper method.

    Before, there were 3 places in the file that called removeChild().
    This was fine, but they now need to call the mutation callback.
    So abstract these into remove_child().
    Also, call the mutation callback in AMP_Video_Sanitizer.
    Ryan Kienstra committed Jan 28, 2018
    Configuration menu
    Copy the full SHA
    4714062 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2018

  1. Issue #843: Initial registration of the REST endpoint for validation.

    Per Weston's description in PR #912,
    It allows sending a POST with markup for validation.
    The headers should have 'Content-Type' of 'application/json.'
    And it should pass the markup in the param 'markup.'
    The current response only has 'is_error.'
    @todo: look at returning more in the response,
    like the stripped tags and attributes.
    Also, add nonce verification.
    Ryan Kienstra committed Jan 29, 2018
    Configuration menu
    Copy the full SHA
    e209005 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2018

  1. Issue #864: Support <amp-carousel> in 'Gallery' widget.

    There's an existing handler to create 'amp-carousel' elements:
    class AMP_Gallery_Embed_Handler.
    So override the 'Gallery' widget class.
    And use that in render_media().
    Otherwise, that function is copied from the parent.
    It calls gallery_shortcode() at the end.
    Which doesn't have a filter for the markup.
    Ryan Kienstra committed Jan 30, 2018
    Configuration menu
    Copy the full SHA
    aed76c6 View commit details
    Browse the repository at this point in the history
  2. Issue #843: Report removed attributes and nodes in a histogram.

    This is only one approach.
    But for now, the response has counts for:
    'removed_nodes' and 'removed_attributes'.
    If a <script> is removed, 'removed_nodes' will be:
    {"script":1}.
    The count will increment every time the same node type is removed.
    There is a similar histogram for 'removed_attributes'.
    Ryan Kienstra committed Jan 30, 2018
    Configuration menu
    Copy the full SHA
    30c666f View commit details
    Browse the repository at this point in the history
  3. Revert "Issue #864: Support <amp-carousel> in 'Gallery' widget."

    This reverts commit aed76c6.
    Ryan Kienstra committed Jan 30, 2018
    Configuration menu
    Copy the full SHA
    76b0f17 View commit details
    Browse the repository at this point in the history
  4. Issue #843: Align equals signs vertically.

    In response to Travis errors.
    @todo: apply next requirement in PR #912.
    Ryan Kienstra committed Jan 30, 2018
    Configuration menu
    Copy the full SHA
    0b7c3fc View commit details
    Browse the repository at this point in the history
  5. Issue #843: Prepare to add headers to frontend GET requests.

    Abstract the logic for the response into get_response().
    This enables using it for the existing REST API logic,
    And the new use-case of full-page GET requests.
    Ryan Kienstra committed Jan 30, 2018
    Configuration menu
    Copy the full SHA
    71744e5 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    da408bf View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2018

  1. Issue #864: Validation data in the response header.

    In a frontend GET request, add a header:
    'AMP-Validation-Error'.
    This outputs whether the sanitizers stripped nodes or tags.
    A possible output is:
    '{"has_error":true,"removed_nodes":{"script":1},"removed_attributes":{"async":1}}'
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    ab3909a View commit details
    Browse the repository at this point in the history
  2. Issue #864: Remove an extra conditional, nest the 'mutation_callback.'

    As Weston mentioned, the child could get the parentNode.
    So there's no reason for the elseif.
    Also, this makes it possible to nest the 'mutation_callback.'
    So it's only called if there's a removal.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    90090b0 View commit details
    Browse the repository at this point in the history
  3. Issue #864: Rename function to finish_output_buffering().

    This function has the same logic as the current get_buffer().
    But the name is more descriptive.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    83dac87 View commit details
    Browse the repository at this point in the history
  4. Issue #843: Remove the extra variabl in the @return tag.

    The return value is simply void.
    So there's no need for any more information.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    70521ef View commit details
    Browse the repository at this point in the history
  5. Issue #843: Add processed markup to REST API response.

    Respond with the markup that is submitted in the request,
    In the value 'processed_markup'.
    Full-page requests won't have the markup in the response.
    esc_html() might not be the best way to escape the markup.
    But it doesn't display properly without escaping.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    cc4fe85 View commit details
    Browse the repository at this point in the history
  6. Issue #843: Output the 'processed_markup' at the bottom of the response.

    Before, it was at the top.
    The most important information seems to be whether there's an error.
    So keep that at the top.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    0b6a4ee View commit details
    Browse the repository at this point in the history
  7. Issue #843: Merge in 'develop' branch and resolve conflicts.

    There were 2 conflicting files.
    The conflicts in amp.php were trivial.
    But there was a change in attribute removal in:
    AMP_Tag_And_Attribute_Sanitizer::sanitize_disallowed_attributes()
    Keep that change from the develop branch,
    But also add the mutation callback.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    db49a33 View commit details
    Browse the repository at this point in the history
  8. Issue #843: Apply validation to post update on wp-admin/post.php.

    Use the nonce on that action to verify this.
    But there will probably be other cases where we'll need validation.
    And therefore other nonce solutions.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    c92bee4 View commit details
    Browse the repository at this point in the history
  9. Issue #843: Verify the nonce before validating on 'save_post'.

    Before, there was no verification.
    So use the new method authorized_nonce() to check for this.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    6b5593b View commit details
    Browse the repository at this point in the history
  10. Issue #843: Sanitize $_GET value in addition to the 'ignore' comment.

    Use sanitize_text_field( wp_unslash() ).
    WPCS still requires nonce verification.
    But there is no nonce on this page.
    This is using the 'edit_form_top' action.
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    a168a5c View commit details
    Browse the repository at this point in the history
  11. Issue #843: Correct Travis error by changing error text.

    I had changed this to 'this content fails...'
    But change it back to 'this post fails...'
    Ryan Kienstra committed Feb 1, 2018
    Configuration menu
    Copy the full SHA
    a52ae05 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2018

  1. Issue #843: Report node removal in the rest of the sanitizers.

    Call the 'mutation_callback' in the remaining sanitizers on node removal.
    But not in the style sanitizer, as they're merely added to 'style-custom'.
    Move remove_child() into AMP_Base_Sanitizer,
    so all of the sanitizers can use it.
    Ryan Kienstra committed Feb 4, 2018
    Configuration menu
    Copy the full SHA
    0472a33 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2018

  1. Merge branch 'develop' of https://github.com/Automattic/amp-wp into a…

    …dd/sanitization-reporting
    westonruter committed Feb 5, 2018
    Configuration menu
    Copy the full SHA
    6d2350f View commit details
    Browse the repository at this point in the history
  2. Issue #843: Call the 'mutation_callback' for more attribute removals.

    Call this in most of the classes that have $node->removeAttr().
    And abstract this into AMP_Base_Sanitizer::remove_attr().
    The style sanitizer does not track removing 'style' attribtues.
    AMP_Style_Sanitizer::process_styles() handles these.
    And they are eventually added to <style amp-custom>.
    Ryan Kienstra committed Feb 5, 2018
    Configuration menu
    Copy the full SHA
    3de38b8 View commit details
    Browse the repository at this point in the history
  3. Issue #843: Merge in develop, resolve conflicts.

    There were conlicts in:
    AMP_Theme_Support and Test_Class_AMP_Theme_Support.
    Resolve them in favor of this feature branch:
    add/sanitization-reporting
    Ryan Kienstra committed Feb 5, 2018
    Configuration menu
    Copy the full SHA
    46e7084 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2018

  1. Issue #843: Merge in develop again, resolve conflicts.

    In most cases, resolve in favor of this branch:
    add/sanitization-reporting
    In some cases, there were deletion in develop.
    Like injecting styles.
    So resolve in favor of those.
    Ryan Kienstra committed Feb 7, 2018
    Configuration menu
    Copy the full SHA
    972ac2c View commit details
    Browse the repository at this point in the history
  2. Issue #843: Rename test to 'get_buffer' instead of 'prepare_response'.

    The function prepare_response() doesn't exist.
    Instead, use get_buffer().
    Ryan Kienstra committed Feb 7, 2018
    Configuration menu
    Copy the full SHA
    d2ab9ee View commit details
    Browse the repository at this point in the history
  3. Issue #843: Fix an issue in the error message.

    Before, the error message always appeared.
    This is because it only checked that the response
    had a value for 'has_error'.
    But this needs to be true in order for there to be a reported error.
    Ryan Kienstra committed Feb 7, 2018
    Configuration menu
    Copy the full SHA
    cb0cfc9 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2018

  1. Issue #843: Revert renaming of methods, adjust unit tests.

    I had renamed some methods in this branch: add/sanitization-reporting.
    Also, remove the parameter from finish_output_buffering().
    That function in the 'develop' branch no longer has as parameter.
    Ryan Kienstra committed Feb 8, 2018
    Configuration menu
    Copy the full SHA
    084acaa View commit details
    Browse the repository at this point in the history
  2. Issue #843: Remove special characters, update documentation.

    There were different characters in prepare_response(),
    Mabye from copying from GitHub.
    Also, adjust documentation, and add a @codingStandardsIgnoreEnd.
    Ryan Kienstra committed Feb 8, 2018
    Configuration menu
    Copy the full SHA
    13ab280 View commit details
    Browse the repository at this point in the history
  3. Issue #843: Change @const to @var for constants.

    As Thierry mentioned,
    this is required for a valid PHPDoc.
    Ryan Kienstra committed Feb 8, 2018
    Configuration menu
    Copy the full SHA
    f996673 View commit details
    Browse the repository at this point in the history
  4. Issue #843: Rename class to 'AMP_Validation_Utils'

    This was previously 'AMP_Mutation_Utils'
    The new name describes better what this does.
    Ryan Kienstra committed Feb 8, 2018
    Configuration menu
    Copy the full SHA
    f8aeca8 View commit details
    Browse the repository at this point in the history
  5. Issue #843; Use constants instead of string literals.

    On Thierry's suggestion,
    As these were already stored in constants.
    Ryan Kienstra committed Feb 8, 2018
    Configuration menu
    Copy the full SHA
    69317b8 View commit details
    Browse the repository at this point in the history
  6. Issue #843: Add nonce verification for the editor message.

    Use check_admin_referer(),
    as this will display the 'are you sure' message.
    Also , update the test.
    Ryan Kienstra committed Feb 8, 2018
    Configuration menu
    Copy the full SHA
    583a6bc View commit details
    Browse the repository at this point in the history
  7. Issue #843: Align comments in addition to variable names.

    In PHPDoc blocks, most of the comments weren't aligned.
    The types aren't aligned.
    Ryan Kienstra committed Feb 8, 2018
    Configuration menu
    Copy the full SHA
    cd910ff View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    a6d071a View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2018

  1. Only report mutations when node/attribute is removed due to invalidity

    * Skip reporting iframe removal when merely being moved
    * Skip reporting removal of form[action] attribute when transformed to action-xhr.
    * Rename sanitizer base methods to make explicit they are for removal of invalid nodes.
    westonruter committed Feb 9, 2018
    Configuration menu
    Copy the full SHA
    b40729b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9954c11 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    15d9186 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    bb7a175 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    29caa8a View commit details
    Browse the repository at this point in the history