Skip to content

Provisionally Afanasy Pools Support. It's not the official implementation. ⭐ 😎

License

Notifications You must be signed in to change notification settings

laurence-trippen/Afanasy-Pools-Support

Repository files navigation

Afanasy Pools Support

A provisionally pools implementation for the open-source render manager CGRU.
It's not the official implementation. It's more of a workaround.

With Afanasy Pools Support you can create pools and assign clients to them.
When you create an Afanasy job, whether using the AfStarter or a software plugin, you can
always specify a pool for rendering.

The pools are more a superficial solution and are not stored in the Afanasy's code, so you don't
see any pools in the Afanasy Web GUI or in the Afanasy Qt GUI.

Why this project?

On the CGRU Roadmap a pool support is planned for the future.
But since we needed a provisional pool solution in one project, we developed
one ourselves which works through a workaround.

The project still has its weaknesses in some places, because it has been developed rapidly.
If there are bugs, just create an issue on Github.

How does it works?

The implementation consists of four important components:

1. MongoDB database

All pools and their associated clients are stored in the MongoDB database.

2. Afanasy Pool Manager

With the Afanasy Pool Manager you can create, edit and delete pools in the MongoDB database. Clients can also be added to or removed from these pools.

3. Pool selection at job submssion

In CGRU, render jobs can be sent to the server in various ways. Until now it is planned that pools can be specified in the AfStarter and Blender plugin. These plugins and programs also access the MongoDB database directly or via. Afanasy Pool Server.

4. Afanasy Pool Server

The Afanasy Pool Server is for plugins that cannot directly access the MongoDB database, but can establish a network connection.

For example, Blender provides Python to create plugins. To access the MongoDB (Pool Database) with Blender Python you need the Python Module PyMongo. Since it is very difficult to install the module on a RenderFarm on every computer in Blender, the Afanasy Pool Server offers an easy way to access the pool data.

The server acts as a mediator between MongoDB and the plugins.

Most plugin environments of 3D programs allow the establishment of network connections.

The Afanasy Pool Server, is a simple TCP server that offers a simple request/response protocol like HTTP. The protocol is stateless and is closed after each response.

The data is sent unencrypted as text over the network. Since most render farms are in the local network, this should not be a problem.

But for the future SSL/TLS support is planned.

Structure as Schema

Pool emulation by hosts exclude mask.

For every Afanasy render job you can specify a hosts exclude mask. In this mask individual render clients can be excluded from a job by a regular expression.

Let's assume we have a render farm with five render clients.

pc-01, pc-02, pc-03, pc-04 and pc-05

With this regular expression, we can, for example, exclude these three render clients from a job.

pc-01|pc-02|pc-04

This means that the job is executed on the two render clients pc-03 and pc-05.

The hosts exclude mask can be used to create pools indirectly. For example, let's take the five render clients from above. Let's define a pool where pc-01 and pc-03 are contained.

From this pool the following mask would have to be generated, so that on the Render Farm, the job can only be executed with pc-01 and pc-03.

Generated mask to run job on renderfarm with pc-01 and pc-03:

pc-02|pc-04|pc-05

This simple algorithm is used to generate an excluded client list from the included pool clients.

def get_excluded_hostnames(all_renderfarm_hostnames, pool_hostnames):
  excluded_hostnames = []
  for hostname in all_renderfarm_hostnames:
    if not hostname in pool_hostnames:
      excluded_hostnames.append(hostname)
  return excluded_hostnames

Software Integration

Features & Future plans

General

  • Easy installation
  • PyMongo installation script for afanasy's embedded Python.

Pool Manager

  • Pool Manager is startable from Keeper.
  • Create, Edit, Delete pools.
  • Add/Remove clients to pools.
  • Add online and offline afanasy clients to pool.
  • Network scan

Software integration

  • Blender plugin pool integration.
  • AfStarter pool integration.
  • Natron pool integration.
  • Fusion pool integration.

Pool server

  • Pool Server implementation. (PyMongo is no longer needed in plugins.)
  • Pool Server simple configurator.
  • Pool Server SSL/TLS Support.

Installation

Coming soon.

Showcase

Start Afanasy Pool Manager from Keeper

Afanasy Pool Manager

Blender Pool Integration

AfStarter Pool Integration

About

Provisionally Afanasy Pools Support. It's not the official implementation. ⭐ 😎

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published