Making IceStorm topic/publisher endpoints explicit

[tweewo]

Goodmorning!

Serverside, we have an IceGrid that runs an IceStorm (version 3.4, sorry for that).

An outside client is not allowed to communicate with the IceGrid (firewall). However, that client can reach the IceStorm's topic manager, using the client's config file (and resolved in code):
IceStorm.TopicManager.Proxy=IceStorm/TopicManager:tcp -p 12998 -h serverhost

However, once the code calls for (C#), :

    TopicPrx topic = createTopic ("db_9800");    // TopicManager retrieves/creates
    ObjectPrx publisher = topic.getPublisher();

The topic proxy appears to be an indirect proxy, the VS debugger shows...
{IceStorm/topic.db_9800 -t -e 1.0 @ IceStorm.IceStorm.IceStorm.TopicManager}

This is not resolvable without the help of IceGrid, so a NoEndpointException is thrown.

Here come the questions...

1. Can I somehow configure the TopicManager to send out explicit endpoints? I tried using PublishedEndpoints in the IceGrid definition, but it does not seem to make a difference:

  <service name="IceStorm" entry="IceStormService,34:createIceStorm" >
    <dbenv name="IceStorm"/>
    <adapter name="IceStorm.TopicManager" endpoints="tcp -h serverhost -p 12998">
      <object identity="IceStorm/TopicManager" type="::IceStorm::TopicManager"/>

      <!-- PublishedEndpoints: attempt to solve the issue -->
      <property name="PublishedEndpoints" value="tcp -h serverhost -p 12998"/>
    </adapter>

    <adapter name="IceStorm.Publish" endpoints="tcp -h serverhost -p 12997">

      <!-- And, will eventually need it here again... -->
      <property name="PublishedEndpoints" value="tcp -h serverhost -p 12997"/>
    </adapter>
  </service>

2. Alternatively, is there perhaps a way that I can make explicit the topic proxy's details in the client's config file, something like this (I have been trying various syntax and failing):
   IceStorm.topic.db_9800=IceStorm/topic.db_9800:tcp -p 12998 -h serverhost

3. Worst option, is there a way to change the client's code to request for an explicit proxy from the topic manager? I couldn't find this in the docs. However, as this is a legacy environment, I prefer not resolve it in the code. It would be ideal if we can configure it somehow :)

Finally, the client will eventually need a publisher - so can we apply the same solution to ensure the publisher's endpoints are explicit?

As an encore ;), and not essential; in an attempt to reduce the port-footprint, is there a way to have the publisher object use the same adapter as the topic manager? Remember we have deployed version 3.4 (sorry again).

Thanks for your help, and for your great products all these years,
Erik

[bernardnormier]

Hi Erik,

1. Can I somehow configure the TopicManager to send out explicit endpoints?

Yes, but not with this approach.

When you set PublishedEndpoints on an indirect object adapter (and all object adapters configured with an IceGrid adapter descriptor are indirect), you configure the endpoints that the object adapter registers with IceGrid upon activation. These endpoints are then returned when you resolve the adapter ID with IceGrid.

When you call create on your TopicManager, it returns a proxy created with "createProxy":

ice/cpp/src/IceStorm/TopicI.cpp

Line 916 in 646140b

prx = _instance->topicAdapter()->createProxy(_id);

This proxy is an indirect proxy when the object adapter is indirect.

The solution here is to not use adapter descriptors to configure your IceStorm object adapters. You should instead configure them with plain properties. This way, these object adapters are "direct" and not known to IceGrid at all:

<service name="IceStorm" entry="IceStormService,34:createIceStorm" >
    <dbenv name="IceStorm"/>
    <!-- no adapter description at all -->
    <property name="IceStorm.TopicManager.Endpoints" value ="tpc -h serverhost -p 12998"/> <!-- direct adapter -->
    ...

With this setup, you can still manage IceStorm from IceGrid (in particular start and stop the icebox server with the IceGrid GUI). However, you can't resolve indirect proxies implemented by IceStorm since IceStorm is now fully "direct".

(We verified this works with Ice 3.7; there is a good chance it also works with 3.4, but I could be missing something).

2. Alternatively, is there perhaps a way that I can make explicit the topic proxy's details in the client's config file

No, there is no way to overwrite a proxy in a config file.

One possibility would be to implement your own trivial Locator for this client and then configure the client to use this Locator (with Ice.Default.Locator). You could even make this trivial Locator a plug-in. But that's of course much more work than the solution in "1".

The Locator in question would resolve IceStorm.IceStorm.IceStorm.TopicManager into tcp -p 12998 -h serverhost.

3. Worst option, is there a way to change the client's code to request for an explicit proxy from the topic manager?

If you can change the code, there are lots of options . But you can't request a direct proxy from the topic manager: when the underlying object adapter is indirect, the createProxy API that IceStorm calls always returns an indirect proxy.

As an encore ;), and not essential; in an attempt to reduce the port-footprint, is there a way to have the publisher object use the same adapter as the topic manager?

No, two object adapters listening on the same interface with tcp cannot share a port. Since IceStorm has 2 object adapters (3 in Ice 3.7), each needs its own port. And there is no way to tell IceStorm "please merge your object adapters".

With Glacier2, you could reduce this external port usage to a single port and use a standard IceGrid deployment for IceStorm (with indirect object adapters):

flowchart LR
    subgraph server [Server side]
        direction LR
        glacier2 -- ports 12998-12999 --> icestorm --> subscriber
        glacier2 --> icegrid
        icestorm --> icegrid
    end
    publisher -- port 4064 --> glacier2 

Loading

but this would require big code updates to your client. If you take this route, I'd recommend upgrading this client to Ice 3.7.

Hope this helps!
Bernard

[tweewo]

Thanks, Bernard, for your very detailed answer.

I can confirm that your solution (1) works very well with 3.4. It is clean, fully configurable, and requires no code changes. The publisher endpoint also becomes explicit using the same technique. Perfect.

The local locator is a very smart approach. I had not thought about, and it would work perfectly for this case. But option (1) is less work :)

Thanks also for the creative Glacier2 solution to port merging! We may yet use that solution in future projects. For the current one, we will just open 2 ports :)

Problem solved! Thanks so much.

Cheers,
Erik