api.content.create replaces archetype-content title with site title during Plone site creation

[datakurre]

How to reproduce:

1. Use plone.api.content.create to create content in "setuphandlers.py" (or custom GS profile configuration step).
2. Create a new Plone site and select the profile in question to be installed for the new site.

Then

Each new Archetype-based content object created with plone.api.content.create will have site title as its title.

Why

1. plone.api.content.create calls Archetypes' processForm-method
2. processForm-method tries to re-fill content fields from request
3. title-widget finds a title field from the request (because the default form for creating a new Plone site has a title field)

Our local fix is to pop "title" key from the request.form before running code with plone.api.content.create.

But note that this is not limited to the site creation from but any values in request.form may cause similar surprises with api.content.create.

If it's not a feature to read field values for plone.api.content.create from request, then maybe request.form should somehow be cleared before creating Archetypes-content and restored afterwise.

[zupo]

Hey @datakurre I'm trying to reproduce this without success. Can you provide more info please:

• snippet of code in your setuphandlers.py
• plone and plone.api versions
• possibly a test case exhibiting this bug?

[hvelarde]

I can confirm the same issue:

• Plone 4.3.2
• plone.api 1.1.0

here's the code I'm using:

SITE_STRUCTURE = (
    dict(type='Folder', title=u'One'),
    dict(type='Folder', title=u'Two'),
    dict(type='Folder', title=u'Three'),
    dict(type='Folder', title=u'Four'),
    dict(type='Folder', title=u'Five'),
)

def create_new_portal_structure(site):
    """Create new site structure as defined.
    """
    logger.info('Creating contet.')
    for item in SITE_STRUCTURE:
        title = item['title']
        id = idnormalizer.normalize(title, 'en')
        if not hasattr(site, id):
            # by default, the whole structure is excluded from navigation
            item['excludeFromNav'] = True
            api.content.create(site, **item)
            logger.info(u'{0} created'.format(title))
        else:
            logger.warn(u'skipping {0}; content already exist'.format(title))

I wrote this test to demonstrate the issue:

    def test_new_portal_structure_titles(self):
        """XXX: plone.api bug: https://github.com/plone/plone.api/issues/99
        """
        for item in SITE_STRUCTURE:
            title = item['title']
            id = idnormalizer.normalize(title, 'en')
            self.assertEqual(self.portal[id].title, title)

but then it gets worst: test pass but running the code in the instance fails: the title of all items is inherited from the portal: 'Site'.

as a workaround I'm using the following:

            obj = api.content.create(site, **item)
            # XXX: following two lines are a workaround for issue in plone.api
            #      see: https://github.com/plone/plone.api/issues/99
            obj.setTitle(title)
            obj.reindexObject('Title')

[tdesvenain]

I get the same problem, but using invokeFactory

On Fri, Oct 18, 2013 at 4:34 PM, Héctor Velarde notifications@github.comwrote:

I can confirm the same issue:

• Plone 4.3.2
• plone.api 1.1.0

here's the code I'm using:

SITE_STRUCTURE = (
dict(type='Folder', title=u'One'),
dict(type='Folder', title=u'Two'),
dict(type='Folder', title=u'Three'),
dict(type='Folder', title=u'Four'),
dict(type='Folder', title=u'Five'),)
def create_new_portal_structure(site):
"""Create new site structure as defined. """
logger.info('Creating contet.')
for item in SITE_STRUCTURE:
title = item['title']
id = idnormalizer.normalize(title, 'en')
if not hasattr(site, id):
# by default, the whole structure is excluded from navigation
item['excludeFromNav'] = True
api.content.create(site, **item)
logger.info(u'{0} created'.format(title))
else:
logger.warn(u'skipping {0}; content already exist'.format(title))

I wrote this test to demonstrate the issue:

def test_new_portal_structure_titles(self):
    """XXX: plone.api bug: https://github.com/plone/plone.api/issues/99        """
    for item in SITE_STRUCTURE:
        title = item['title']
        id = idnormalizer.normalize(title, 'en')
        self.assertEqual(self.portal[id].title, title)

but then it gets worst: test pass but running the code in the instance
fails: the title of all items is inherited from the portal: 'Site'.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/99#issuecomment-26600331
.

Thomas Desvenain

Téléphone : 09 51 37 35 18

[hvelarde]

seems to me this is an Archetypes bug; any reference?

[hvelarde]

same issue happens with Dexterity-based content types.

I just discovered a new thing: this behavior does not occurs on tests, only when running the instance.

[mattss]

It seems like the processForm call from api.content.create is an easy but dangerous way to finalise the archetype content, is it will pull any matching field values from the request and override those passed into api.content.create. Can we take what we need from processForm instead (rename after creation and relevant events)?

[jaroel]

What needs to be done, and who is going to do it?

[pgrunewald]

I am also interested in getting this issue solved.

Maybe we could use the values parameter of processForm and populate it with dictionary containing title (if present) and all other passed data from kwargs. That way the request object is not consulted, when retrieving potential form data.

[david-batranu]

The changes introduced in f73bddc breaks content creation when other fields are involved. Consider this usecase:

from datetime import datetime

from Products.Archetypes import atapi
from Products.ATContentTypes.content import schemata
from Products.ATContentTypes.content import base

from plone import api


class IMyContent(Interface):
    """ """

MyContentSchema = schemata.ATContentTypeSchema.copy() + atapi.Schema((

    atapi.DateTimeField(
        'date',
        schemata='default',
        required=True,
        languageIndependent=True,
    ),
)


class MyContent(base.ATCTContent):
    implements(IMyContent)

    meta_type = 'MyContent'
    schema = MyContentSchema


atapi.registerType(MyContent, 'my.package')

def do_manual_creation(context):
    fields = {'title': 'A title', 'date': datetime.now()}
    content = api.content.create(context, 'MyContent', id='mycontent', **fields)

    # pre 1.6.1:
    # >>> type(content.date)
    # <class 'DateTime.DateTime.DateTime'>

    # post 1.6.1
    # >>> type(content.date)
    # <type 'NoneType'>

This is because CalendarWidget.process_form is called with fields as form. That method does some insane things, completely ignoring the passed value and finally returning '' which gets saved by DateTimeField as None.

Pre 1.6.1, form was request.form, set by BaseObject._processForm, so in the abve usecase, date would be completely missing from the form, resulting in empty_marker to be returned by the widget, and the field to not be mutated; the value set by invokeFactory preserved.

I believe a better solution for plone.api.content.create would be to replace processForm(values=kwargs) with processForm(values={}). From what I can see, widgets mostly do the right thing and return empty_marker if field.getName() is missing from the request. Perfect when programatically adding content through invokeFactory/plone.api.content.create as the set values don't get modified.

I will make a pull request, with the change.

EDIT: it's actually {None: None} as {} is falsy.

[gotcha]

@david-batranu Did you check that the bug with title mentioned above by @datakurre is still fixed ?

[david-batranu]

@gotcha you're right! I fixed the pull request, it now uses a {None: None} dict, so it will pass the check in BaseObject._processForm.