customActions from plone.supermodel are being called before all config are done when using plone.app.testing

[idgserpro]

This happened in collective.cover's PR after Plone 4.3.12 release.

Using Plone 4.3.11, collective.cover tests were working. They stopped working in 4.3.12. We detected (by doing a pdb in supermodel finalizeSchema) that the cause is cover's supermodel fields that aren't "loaded". supermodel needs to execute a customAction after all zcml are loaded. For some zcml dependency we didn't detected yet, in Plone 4.3.12, supermodel is loaded along packages that plone.app.testing loads. At that moment, cover's zcml isn't loaded yet so supermodel's customAction don't consider cover configurations, giving the error.

When bin/instance is called, xmlconfig.file is called only one time. When bin/test is called, xmlconfig.file is called for each package in products. So the customActions are executed multiple times in bin/test making customActions of supermodel being executed before collective.cover's configuration.

We don't know which package in

plone.app.testing/plone/app/testing/layers.py

Line 38 in 62b236e

products = (

caused the error.

[idgserpro]

Investigating a little more, I discovered that isn't the collective.cover that needs to be loaded before the finalizeSchema function is executed, but rather the plone.autoform (which is used by the cover).

I also discovered that the finalizeSchema function is called when the Products.TinyMCE is loaded. At this moment, the plone.autoform isn't loaded yet.

We could just add the plone.autoform into the products variable, before Products.TinyMCE. But that could give trouble in the future, as it did now.

So I suggest add:

<include zcml:condition="installed plone.autoform" package="plone.autoform" />

before this supermodel line. Thus, we ensure that the plone.autoform will always be loaded in the tests, before the finalizeSchema function is called.

@mauritsvanrees @davisagli @hvelarde opinions?

[hvelarde]

I can't help on this; sorry but it's out of my current knowledge.

[mauritsvanrees]

That seems to make sense, and it fixes the problems with the collective.cover tests.
+1 on making that change to include plone.autoform zcml.

[mauritsvanrees]

@idgserpro When I try your fix and run the plone.supermodel tests in coredev 4.3, I get lots of test failures:

        ZopeXMLConfigurationError: File "/Users/maurits/community/plone-coredev/4.3/src/plone.supermodel/plone/supermodel/configure.zcml", line 8.4-9.35
        ZopeXMLConfigurationError: File "/Users/maurits/shared-eggs/plone.autoform-1.6.2-py2.6.egg/plone/autoform/configure.zcml", line 5.4-5.39
        ZopeXMLConfigurationError: File "/Users/maurits/shared-eggs/plone.z3cform-0.8.1-py2.6.egg/plone/z3cform/configure.zcml", line 9.2-9.35
        ZopeXMLConfigurationError: File "/Users/maurits/shared-eggs/plone.z3cform-0.8.1-py2.6.egg/plone/z3cform/templates.zcml", line 12.2
        ConfigurationError: ('Unknown directive', u'http://namespaces.zope.org/browser', u'page')

The zcml file from the last line seems innocent: the directive is there.

So while it seemed okay, this is not the right solution.

[idgserpro]

@mauritsvanrees really weird. Like you said, the directive is there.

When I have the time, I take a look at coredev.

[idgserpro]

@mauritsvanrees the problem is that coredev tests include plone.autoform egg. It is not an isolated test with only plone.supermodel dependencies. Then the condition:

zcml:condition="installed plone.autoform"

will be true and the plone.autoform zcml will be included in the plone.supermodel tests. The test needs to be reviewed considering this.

[idgserpro]

@mauritsvanrees for the directive to work, it's necessary load the Products.Five into the supermodel tests. Then you need to add:

<include package="Products.Five" file="configure.zcml" />

in
test.py
directives.txt
fields.txt
schema.txt

In addition, you need to change the schema.txt test to consider the plone.autoform directives, which are now loaded:

Failure in test /home/user/buildout.coredev/src/plone.supermodel/plone/supermodel/schema.txt
Failed doctest test for schema.txt
  File "/home/user/buildout.coredev/src/plone.supermodel/plone/supermodel/schema.txt", line 0

----------------------------------------------------------------------
File "/home/user/buildout.coredev/src/plone.supermodel/plone/supermodel/schema.txt", line 110, in schema.txt
Failed example:
    print serializeModel(model) # doctest: +NORMALIZE_WHITESPACE
Expected:
    <model xmlns:i18n="http://xml.zope.org/namespaces/i18n" xmlns="http://namespaces.plone.org/supermodel/schema">
      <schema>
        <field name="title" type="zope.schema.TextLine">
          <title>Title</title>
        </field>
        <field name="description" type="zope.schema.Text">
          <description>A short summary</description>
          <min_length>10</min_length>
          <required>False</required>
          <title>Description</title>
        </field>
      </schema>
    </model>
Got:
    <model xmlns:i18n="http://xml.zope.org/namespaces/i18n" xmlns:security="http://namespaces.plone.org/supermodel/security" xmlns:form="http://namespaces.plone.org/supermodel/form" xmlns="http://namespaces.plone.org/supermodel/schema">
      <schema>
        <field name="title" type="zope.schema.TextLine">
          <title>Title</title>
        </field>
        <field name="description" type="zope.schema.Text">
          <description>A short summary</description>
          <min_length>10</min_length>
          <required>False</required>
          <title>Description</title>
        </field>
      </schema>
    </model>

[hvelarde]

here's a screenshot of the actual error:

according to the changelog there's nothing in this package that has changed in order to create that problem; IMO this issue has to be closed and we need to continue looking for what is causing the problem.

[hvelarde]

I describing the issue in dept here: collective/collective.cover#759

I already have a workaround.

[idgserpro]

@hvelarde The problem is already described here, I didn't get it why you opened another one in collective.cover...

@mauritsvanrees can you help us here? We gave two suggestions on how to solve the problem:

#39 (comment)

#39 (comment)

If you don't agree in plone.autoform influencing plone.supermodel tests, can you at least analyse adding plone.autoform before TinyMCE in

plone.app.testing/plone/app/testing/layers.py

Line 72 in 62b236e

('Products.TinyMCE' , {'loadZCML': True}, ),

?

[hvelarde]

@davisagli can you give a clue here? I see you're the father of the creature ;)