[1LP][RFR] Add customer scenario marker/filter, move tier and requirement markers #9225

mshriver · 2019-08-20T19:32:28Z

Change how the tier and requirement markers are registered
Remove unused node annotation logic
Update pytest-polarion-collect and other polarion packages
Add a marker/filter for customerscenario meta in polarion

Changes to test_advanced_search are to repair its metadata collection and to prevent exceptions from pytest-polarion-collect during collection.

mshriver · 2019-08-20T19:33:55Z

conf/polarion_tools.yaml

@@ -66,6 +66,7 @@ docstrings:
        # default for manual mark is 'notautomated'
        # @pytest.mark.manual('manualonly') to set 'manualonly'
        linkedWorkItems: "@pytest.mark.requirements"
+        customerscenario: "@pytest.mark.requirements.customer_stories"


Not possible to set meta with these markers, evaluating if its worth supporting this. Could also be set via BZ marker if by resolving items attached to the BZ.

izapolsk · 2019-08-21T15:59:54Z

it's really great PR !. I love it

mshriver · 2019-08-26T15:26:34Z

I've abstracted the meta-filter itself into pytest-polarion-collect:
https://gitlab.com/mkourim/pytest-polarion-collect/merge_requests/5

sshveta · 2019-08-26T17:56:15Z

cfme/test_requirements.py

+    assignee_id='ndhandre',
+)
+
+day2 = pytest.mark.requirement(


I thought day 2 scenarios are different from customer scenarios . If yes then we need to change the description

So, they are not mutually exclusive.

customerscenario meta is for polarion boolean flag, and may apply to test cases that are not necessarily linked to customer stories/day2 initiatives. It is the lowest level marker (meta) that indicates a test case is linked to a customer use case. It could be applied automatically by the BZ meta marker if the marked BZ is linked to a customer support case.

This requirement marker is used to group/link test cases that are a direct result of the customer stories initiative, or are related specifically to day2 operations type testing.

I chose to change this requirement marker to day2 to more clearly separate the names, and because its shorter. I left the first argument, the requirement title, the same, because that keeps the existing polarion requirement linked. I'll leave it to the requirement owner, @digitronik, if he also wants to change the requirement title.

@sshveta would you like to see more documentation in the marker module for customerscenario?

@mshriver : yes please , day 2 is not very evident by name that it is for customer stories .
@digitronik : What do you think ?

@mshriver as per our discussion is it possible to put same name means customer_stories.
one will be requirement and other will be simple meta marker.

@pytest.mark.customerscenario: sets customerscenario flag (polarion), indicates a test originating from a customer Bugzilla related to specific FA.

@requirement.customer_stories: Customer operations, actual integration tests means sharing multiple FA. It not bound to only customer bugzilla but customer stories collected from

Bugzilla

Mail threads

Support people threads

FA owner thoughts (how specific FA used in combination with other FAs)

@sshveta we will add customerscenario meta over customer stories tests. but customer_stories requirement marker will not applicable for every customer Bugzilla automation test.

Changed it back

mshriver · 2019-08-27T20:21:42Z

cfme/utils/smem_memory_monitor.py

@@ -246,7 +246,7 @@ def get_miq_server_id(self):

    def get_pids_memory(self):
        result = self.ssh_client.run_command(
-            'smem -c \'pid rss pss uss vss swap name command\' | sed 1d')
+            "/usr/bin/python2.7 /usr/bin/smem -c 'pid rss pss uss vss swap name command' | sed 1d")


This is otherwise unrelated, and I can move the commit to a different branch if preferred. Found this running cfme-performance test repository, smem is not python 3 compatible, this is being run on the appliance, not locally.

mshriver · 2019-08-27T20:28:16Z

cfme/tests/webui/test_advanced_search.py

    view = _navigation(param, appliance)
+    assert view.search.is_advanced_search_possible, (f"Advanced search not displayed "


@psimovec I found when updating the polarion tooling packages, that this pattern of decorating to inject the tests was not including metadata.

Even modifying the inject_tests methods to explicitly include the __doc__ for the methodized test function, the meta parsing tools were not processing it correctly. There will need to be updates to these tools to support a test structure like this.

In the mean time I collapsed these two tests, as a separate test just to assert that the advanced search button is there is redundant to a test that is actually asserting that advanced search is opened when clicked.

I left methodize/inject_tests in place, but removed the decorators and put the boilerplate in place so that meta would be applied.

sshveta

LGTM

sshveta · 2019-08-28T18:36:19Z

cfme/tests/v2v/test_csv_import.py

@@ -19,6 +19,7 @@

 pytestmark = [
    test_requirements.v2v,
+    pytest.mark.customer_scenario,


sshveta · 2019-08-28T18:48:31Z

cfme/tests/webui/test_advanced_search.py

+        SearchParam('ansible_tower_job_templates', 'All', 'ansible_tower_explorer_job_templates',
+                    'Job Template (Ansible Tower) : Name',
+                    ('sidebar.job_templates', 'All Ansible Tower Job Templates')),
+


This empty line can be removed

Add customer-scenario filter and marker register the markers default customerscenario update pytest-polarion-collect and other polarion packages Remove test injection from test_advanced_search, for meta The polarion meta was not correctly parsed, and with updates to pytest-polarion-collect and polarion_docstrings, the unmerged docstrings are causing exceptions and failures during collection. Go to boilerplate for this test module, until the meta processing is resolved. Update smem to run with python2 explicitly

john-dupuy

One minor comment about an old TODO, nice PR! 👍

john-dupuy · 2019-08-29T19:12:39Z

cfme/markers/marker_filters.py

+        keep.append(item)
+
+    items[:] = keep
+    # TODO(rpfannsc) add a reason after pytest #1372 is fixed


Can this TODO be removed? I see that the issue referenced is resolved pytest-dev/pytest#1372

mshriver added the enhancement label Aug 20, 2019

mshriver requested a review from izapolsk August 20, 2019 19:32

mshriver commented Aug 20, 2019

View reviewed changes

dajoRH added the lint-ok label Aug 20, 2019

mshriver changed the title ~~Move tier and requirement markers out of fixtures~~ [WIPTEST] Move tier and requirement markers out of fixtures Aug 20, 2019

dajoRH added the WIP-testing label Aug 20, 2019

mshriver changed the title ~~[WIPTEST] Move tier and requirement markers out of fixtures~~ [WIPTEST] Add meta filter, move tier and requirement markers Aug 21, 2019

mshriver force-pushed the polarion-custscenario branch 2 times, most recently from 85b6dc4 to 8ef68e5 Compare August 21, 2019 15:53

mshriver requested a review from digitronik August 21, 2019 15:53

mshriver force-pushed the polarion-custscenario branch from 8ef68e5 to 54d4599 Compare August 21, 2019 16:03

mshriver mentioned this pull request Aug 22, 2019

[1LP][RFR] Vm name enhancement - to append assignee to VM name #9186

Merged

mshriver force-pushed the polarion-custscenario branch 2 times, most recently from 33b2fd0 to fdd2fb6 Compare August 26, 2019 17:40

mshriver changed the title ~~[WIPTEST] Add meta filter, move tier and requirement markers~~ [RFR] Add meta filter, move tier and requirement markers Aug 26, 2019

mshriver changed the title ~~[RFR] Add meta filter, move tier and requirement markers~~ [RFR] Add customer scenario marker/filter, move tier and requirement markers Aug 26, 2019

dajoRH removed the WIP-testing label Aug 26, 2019

mshriver force-pushed the polarion-custscenario branch from fdd2fb6 to 40a193b Compare August 26, 2019 17:49

sshveta suggested changes Aug 26, 2019

View reviewed changes

mshriver changed the title ~~[RFR] Add customer scenario marker/filter, move tier and requirement markers~~ [WIPTEST] Add customer scenario marker/filter, move tier and requirement markers Aug 26, 2019

dajoRH added the WIP-testing label Aug 26, 2019

mshriver force-pushed the polarion-custscenario branch 2 times, most recently from 9b96714 to 4bc1fb9 Compare August 27, 2019 20:19

mshriver commented Aug 27, 2019

View reviewed changes

mshriver changed the title ~~[WIPTEST] Add customer scenario marker/filter, move tier and requirement markers~~ [RFR] Add customer scenario marker/filter, move tier and requirement markers Aug 28, 2019

dajoRH removed the WIP-testing label Aug 28, 2019

mshriver requested a review from lina-is-here August 28, 2019 15:47

mshriver requested a review from sshveta August 28, 2019 15:47

sshveta approved these changes Aug 28, 2019

View reviewed changes

mshriver mentioned this pull request Aug 29, 2019

[WIPTEST] test_advanced_search - doctrings get lost when transforming function to method #9267

Closed

mshriver force-pushed the polarion-custscenario branch from 4bc1fb9 to 07dc41c Compare August 29, 2019 12:29

mshriver changed the title ~~[RFR] Add customer scenario marker/filter, move tier and requirement markers~~ [1LP][RFR] Add customer scenario marker/filter, move tier and requirement markers Aug 29, 2019

john-dupuy approved these changes Aug 29, 2019

View reviewed changes

jawatts approved these changes Aug 30, 2019

View reviewed changes

jawatts merged commit 7f5dacd into ManageIQ:master Aug 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[1LP][RFR] Add customer scenario marker/filter, move tier and requirement markers #9225

[1LP][RFR] Add customer scenario marker/filter, move tier and requirement markers #9225

mshriver commented Aug 20, 2019 •

edited

Loading

mshriver Aug 20, 2019

izapolsk commented Aug 21, 2019

mshriver commented Aug 26, 2019

sshveta Aug 26, 2019

mshriver Aug 26, 2019

mshriver Aug 26, 2019

sshveta Aug 27, 2019

digitronik Aug 27, 2019

mshriver Aug 27, 2019

mshriver Aug 27, 2019

mshriver Aug 27, 2019

sshveta left a comment

sshveta Aug 28, 2019

sshveta Aug 28, 2019

john-dupuy left a comment

john-dupuy Aug 29, 2019

		view = _navigation(param, appliance)
		assert view.search.is_advanced_search_possible, (f"Advanced search not displayed "

[1LP][RFR] Add customer scenario marker/filter, move tier and requirement markers #9225

[1LP][RFR] Add customer scenario marker/filter, move tier and requirement markers #9225

Conversation

mshriver commented Aug 20, 2019 • edited Loading

Choose a reason for hiding this comment

izapolsk commented Aug 21, 2019

mshriver commented Aug 26, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sshveta left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

john-dupuy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mshriver commented Aug 20, 2019 •

edited

Loading