Remote Home-Assistant: Linking multiple instances via Websocket API #13876

lukas-hetzenecker · 2018-04-14T12:35:49Z

Description:

This platform links multiple home-assistant instances together, replacing the deprecated "Multiple Instances".
The master instance connects to the Websocket APIs of the slaves, the connection options are specified via the host, port, and secure configuration parameters. An API password can also be set via api_password.

After the connection is completed, the remote states get populated into the master instance.
The entity ids can optionally be prefixed via the entity_prefix parameter.

The component keeps track which objects originate from which instance. Whenever a service is called on an object, the call gets forwarded to the particular slave instance.

When the connection to the remote instance is lost, all previously published states are removed again from the local state registry.

A possible use case for this is to be able to use different z-wave networks, on different z-wave sticks (with the second one possible running on another computer in a different location). This is a highly requested feature, as can be seen in the following forum postings:

Currently this use case could be solved by using mqtt_eventstream, but this comes with its own set of problems, e.g.: https://community.home-assistant.io/t/master-slave-with-mqtt-eventstream-retain-all-published-devices/17012

Another option would be to use, mqtt_statestream, but this makes dealing with service calls a real hassle (see https://community.home-assistant.io/t/best-way-to-connect-zwave-switches-on-second-ha-instance-to-main-ha-instance/40577 )

My alternative proposes a way, which

does not need any extra configuration stating which remote objects are imported
makes all remote objects available locally
retains all attributes of the remote objects
removes remote objects again after a connection loss to the remote instance
allows interacting with remote objects via service calls
allows customization of those remote objects in the master config in the customize block

.. and therefore makes linking multiple instances really simple.

Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.io#5170

Example entry for `configuration.yaml` (if applicable):

remote_homeassistant:
  slaves:
  - host: localhost
    port: 8124
  - host: localhost
    port: 8125
    api_password: test
    entity_prefix: "slave02_"

Checklist:

The code change is tested and works locally.
Local tests pass with tox. Your PR cannot be merged unless tests pass

If user exposed functionality or configuration variables are added/changed:

Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

New dependencies have been added to the REQUIREMENTS variable (example).
New dependencies are only imported inside functions that use them (example).
New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
New files were added to .coveragerc.

If the code does not interact with devices:

Tests have been added to verify that the new code works.

which connects to a remote Home-Assistant instance via the Websocket API

According to the changelog, they should be compatible ( https://github.com/aaugustin/websockets/blob/master/docs/changelog.rst ) Compression is now enabled by default

cmsimike · 2018-04-14T15:23:17Z

This is huge for me! Thank you!!

balloob · 2018-04-20T14:18:11Z

homeassistant/components/remote_homeassistant.py

+from homeassistant.config import DATA_CUSTOMIZE
+import homeassistant.helpers.config_validation as cv
+
+REQUIREMENTS = ['websockets==4.0.1']


We should use aiohttp to make the websocket connection as it's already part of the core.

balloob · 2018-04-20T14:21:47Z

homeassistant/components/remote_homeassistant.py

+            """Publish remove event on local instance."""
+            if message['type'] == 'result':
+                pass
+            elif message['type'] == 'event':


Use guard clauses to make the code more readable.

elif message['type'] != 'event': return if message…

balloob · 2018-04-20T14:22:44Z

homeassistant/components/remote_homeassistant.py

+SLAVES_SCHEMA = vol.Schema({
+    vol.Required(CONF_HOST): cv.string,
+    vol.Optional(CONF_PORT, default=8123): cv.port,
+    vol.Optional(CONF_SECURE, default=False): cv.boolean,


It would make more sense to have the user just define a url like https://blabla.duckdns.org:1220

wss://blabla.duckdns.org:1220 .. hopefully users won't confuse ws/wss with http/https?

balloob · 2018-04-20T14:23:19Z

.coveragerc

@@ -208,6 +208,9 @@ omit =
    homeassistant/components/raspihats.py
    homeassistant/components/*/raspihats.py

+    homeassistant/components/remote_homeassistant.py


Nope.

This needs like a 1000 tests added.

balloob · 2018-04-20T14:24:50Z

homeassistant/components/remote_homeassistant.py

+
+_LOGGER = logging.getLogger(__name__)
+
+CONF_SLAVES = 'slaves'


Besides slaves being a politically loaded word, it's also the wrong description. We are not delegating work to them, we're integrating them into our instance. I think that we should just call the entry instances

totally agree, fixed

balloob · 2018-04-20T14:26:05Z

homeassistant/components/remote_homeassistant.py

+        except websockets.exceptions.ConnectionClosed:
+            _LOGGER.error('remote websocket connection closed')
+            await self._disconnected()
+            return


Unnecessary return

cmsimike · 2018-05-02T19:15:39Z

Really hype to get this into hass!

balloob · 2018-05-02T20:11:06Z

@cmsimike maybe you can help him with writing some tests?

lukas-hetzenecker · 2018-05-02T20:59:11Z

Wouldn't mind that at all :) - but please tell me before you do, would otherwise start soonish with them

cmsimike · 2018-05-02T21:34:06Z

@balloob thanks for letting me know that's all that's required! i have added it to my todo list. not sure when i can get to it.

@lukas-hetzenecker definitely, but as above, might be a bit before i can get to it.

lukas-hetzenecker · 2018-05-02T21:49:17Z

FYI, just started with the test cases.. (preparation for my PyDays talk can wait.. ;) )

…onents" This reverts commit 6c8e23d1b2001e027fd03593d5010f8d7b479610.

houndci-bot · 2018-05-14T18:46:47Z

tests/components/test_remote_homeassistant.py

+
+        #self.hass_main.block_till_done()
+        pass
+


blank line at end of file

houndci-bot · 2018-05-14T18:46:47Z

tests/components/test_remote_homeassistant.py

+        #setup_component(self.hass_main, remote_homeassistant.DOMAIN, {
+        #    remote_homeassistant.DOMAIN: config})
+
+        #self.hass_main.block_till_done()


block comment should start with '# '

houndci-bot · 2018-05-14T18:46:47Z

tests/components/test_remote_homeassistant.py

+        #            }
+        #        ]
+        #}
+        #setup_component(self.hass_main, remote_homeassistant.DOMAIN, {


block comment should start with '# '

houndci-bot · 2018-05-14T18:46:47Z

tests/components/test_remote_homeassistant.py

+        #                #remote_homeassistant.CONF_PORT: instance01_port
+        #            }
+        #        ]
+        #}


block comment should start with '# '

houndci-bot · 2018-05-14T18:46:48Z

tests/components/test_remote_homeassistant.py

+        #self.hass_instance01.block_till_done()
+
+
+        #config = {


too many blank lines (2)
block comment should start with '# '

houndci-bot · 2018-05-14T18:46:48Z

tests/components/test_remote_homeassistant.py

+        #    }
+        #)
+
+        #setup_component(self.hass_instance01, 'websocket_api')


block comment should start with '# '

houndci-bot · 2018-05-14T18:46:48Z

tests/components/test_remote_homeassistant.py

+        #            http.CONF_SERVER_PORT: instance01_port,
+        #        }
+        #    }
+        #)


block comment should start with '# '

houndci-bot · 2018-05-14T18:46:48Z

tests/components/test_remote_homeassistant.py

+
+        #self.hass_main.block_till_done()
+
+        #setup_component(


block comment should start with '# '

houndci-bot · 2018-05-14T18:46:49Z

tests/components/test_remote_homeassistant.py

+
+        self.hass_main = get_test_home_assistant()
+
+        #self.hass_main.block_till_done()


block comment should start with '# '

houndci-bot · 2018-05-14T18:46:49Z

tests/components/test_remote_homeassistant.py

+    def setup_method(self):
+        """Setup things to be run when tests are started."""
+        self.hass_instance01 = get_test_home_assistant()
+        #self.hass_instance02 = get_test_home_assistant()


block comment should start with '# '

MartinHjelmare · 2018-05-19T11:21:42Z

homeassistant/components/remote_homeassistant.py

+
+CONFIG_SCHEMA = vol.Schema({
+    DOMAIN: vol.Schema({
+        vol.Required(CONF_INSTANCES): [INSTANCES_SCHEMA],


vol.Required(CONF_INSTANCES): vol.All(cv.ensure_list, [INSTANCES_SCHEMA]),

MartinHjelmare · 2018-05-19T11:22:50Z

homeassistant/components/remote_homeassistant.py

+    """Set up the remote_homeassistant component."""
+    conf = config.get(DOMAIN)
+
+    for instance in conf.get(CONF_INSTANCES):


Don't use dict.get(key) for required config items, use dict[key].

MartinHjelmare · 2018-05-19T11:36:21Z

homeassistant/components/remote_homeassistant.py

+    vol.Optional(CONF_SECURE, default=False): cv.boolean,
+    vol.Optional(CONF_API_PASSWORD): cv.string,
+    vol.Optional(CONF_SUBSCRIBE_EVENTS,
+                 default=DEFAULT_SUBSCRIBED_EVENTS): cv.ensure_list,


vol.All(cv.ensure_list, [cv.string])

MartinHjelmare · 2018-05-19T11:37:28Z

homeassistant/components/remote_homeassistant.py

+    def __init__(self, hass, conf):
+        """Initialize the connection."""
+        self._hass = hass
+        self._host = conf.get(CONF_HOST)


self._host = conf[CONF_HOST]

MartinHjelmare · 2018-05-19T11:39:07Z

homeassistant/components/remote_homeassistant.py

+
+    async def async_connect(self):
+        """Connect to remote home-assistant websocket..."""
+        url = '%s://%s:%s/api/websocket' % (


Use new style string formatting: https://pyformat.info/.

Maybe break out url string into constant?

MartinHjelmare · 2018-05-19T11:58:14Z

homeassistant/components/remote_homeassistant.py

+            if self._entity_prefix:
+                domain, object_id = split_entity_id(entity_id)
+                object_id = self._entity_prefix + object_id
+                entity_id = domain + '.' + object_id


Use new style string formatting.

MartinHjelmare · 2018-05-19T12:00:43Z

homeassistant/components/remote_homeassistant.py

+        asyncio.ensure_future(self._recv())
+
+    def _next_id(self):
+        _id = self.__id


Add docstring.

MartinHjelmare · 2018-05-19T12:00:49Z

homeassistant/components/remote_homeassistant.py

+        return _id
+
+    async def _call(self, callback, message_type, **extra_args):
+        _id = self._next_id()


Add docstring.

MartinHjelmare · 2018-05-19T12:06:47Z

homeassistant/components/remote_homeassistant.py

+                def _remove_prefix(entity_id):
+                    domain, object_id = split_entity_id(entity_id)
+                    object_id = object_id.replace(self._entity_prefix, '', 1)
+                    return domain + '.' + object_id


Use new style string formatting.

MartinHjelmare · 2018-05-19T12:12:53Z

tests/components/test_remote_homeassistant.py

+instance01_port = 8124
+instance02_port = 8125
+
+class TestRemoteHomeassistant(object):


I suggest you go with pytest tests that are not class based. That will be cleaner and you probably want to use the async home assistant api, so then write tests that are coroutines.

houndci-bot · 2018-05-19T22:09:03Z

homeassistant/components/remote_homeassistant.py

+from homeassistant.helpers.typing import HomeAssistantType, ConfigType
+from homeassistant.const import (CONF_HOST, CONF_PORT, EVENT_CALL_SERVICE,
+                                 EVENT_HOMEASSISTANT_STOP,
+                                 EVENT_STATE_CHANGED, EVENT_SERVICE_REGISTERED, CONF_URL)


line too long (89 > 79 characters)

houndci-bot · 2018-05-19T22:09:03Z

homeassistant/components/remote_homeassistant.py

+import homeassistant.components.websocket_api as api
+from homeassistant.core import EventOrigin, split_entity_id
+from homeassistant.helpers.typing import HomeAssistantType, ConfigType
+from homeassistant.const import (CONF_HOST, CONF_PORT, EVENT_CALL_SERVICE,


'homeassistant.const.CONF_URL' imported but unused

houndci-bot · 2018-05-19T22:09:03Z

homeassistant/components/remote_homeassistant.py

+from homeassistant.helpers.typing import HomeAssistantType, ConfigType
+from homeassistant.const import (CONF_HOST, CONF_PORT, EVENT_CALL_SERVICE,
+                                 EVENT_HOMEASSISTANT_STOP,
+                                 EVENT_STATE_CHANGED, EVENT_SERVICE_REGISTERED, CONF_URL)


line too long (89 > 79 characters)

houndci-bot · 2018-05-19T22:09:03Z

homeassistant/components/remote_homeassistant.py

+import homeassistant.components.websocket_api as api
+from homeassistant.core import EventOrigin, split_entity_id
+from homeassistant.helpers.typing import HomeAssistantType, ConfigType
+from homeassistant.const import (CONF_HOST, CONF_PORT, EVENT_CALL_SERVICE,


'homeassistant.const.CONF_URL' imported but unused

balloob · 2018-05-19T22:30:31Z

So I've just added a stub test to get 2 hass instances work together to help you move along. Because you're going around our async infrastructure, I could not get it working without an asyncio.sleep. You will have to fix this before this can get merged.

However, and this brings me to the next part: the code is messy, too many features for MVP, too configurable and the data flows in the wrong direction.

Functionality like this should preferably proposed in our architecture repo to allow people to discuss such major features.

Here is an MVP that I would accept. Client is the one that has "remote_homeassistant" configured. Master is another HASS instance with the websocket API running.

Name should be updated. Maybe "replicate" ? Remote is confusing with the remote component.
Clients push their states to the "master" (current code pulls from master to client)
Clients can only publish to 1 master
For MVP, clients will only publish their states to the master. Services and other events are out of scope for MVP.
When a client disconnected, all states are removed from the master.

The reason for keeping the config decentralized, not centralized is that it will only require the master instance to have a fixed IP. All other clients can just spin up, publish states while they are online and drop off, and it will be cleaned up.

The other nice thing about this approach is that it can be broken up in multiple PRs:

Add a new "set state" command to the websocket API that will set the state but also remove them when the connection drops.
Add a new component that makes a connection and pushes all states at time of setup and then listens for state changes and pushes them out.

I am going to close this PR now as this functionality should be added in 2 PRs. If you don't agree with my proposal, feel free to open an issue in the architecture repo so we can discuss this.

godinperson · 2018-05-23T19:56:55Z

No, it was so close...

lukas-hetzenecker · 2018-05-23T20:00:51Z

@balloob made some good points though ;)
But, for the time being, you can use it by putting it into your custom_components

trebolcinco · 2018-05-27T16:14:55Z

@lukas-hetzenecker was super simple to setup as a custom component, thanks for doing this. Have a large house spread across a couple buildings and this solves my zwave distance issues.

steinis123 · 2018-08-05T10:11:34Z

Will the be any work done to this great plugin? I had some issues when it comes to stability.

lukas-hetzenecker added 2 commits April 14, 2018 13:51

Implementation of remote_homeassistant, a component

4afc91a

which connects to a remote Home-Assistant instance via the Websocket API

add requirements_all.txt and .coveragerc

5f3b0ff

homeassistant added cla-signed new-platform integration: remote_homeassistant labels Apr 14, 2018

Update websocket module for media_player/webostv and spc components

d8b24a9

According to the changelog, they should be compatible ( https://github.com/aaugustin/websockets/blob/master/docs/changelog.rst ) Compression is now enabled by default

homeassistant added the cla-signed label Apr 14, 2018

require hostname

b1e47cc

lukas-hetzenecker mentioned this pull request Apr 14, 2018

Added documentation for remote_homeassistant home-assistant/home-assistant.io#5170

Closed

2 tasks

balloob reviewed Apr 20, 2018

View reviewed changes

balloob mentioned this pull request May 2, 2018

Create a binary_sensor using the gpiozero library #12105

Closed

9 tasks

lukas-hetzenecker added 2 commits May 14, 2018 20:44

requested updates

7b04b5d

Revert "Update websocket module for media_player/webostv and spc comp…

88f7ad9

…onents" This reverts commit 6c8e23d1b2001e027fd03593d5010f8d7b479610.

homeassistant added the cla-signed label May 14, 2018

houndci-bot reviewed May 14, 2018

View reviewed changes

MartinHjelmare changed the title ~~Remote Home-Assistance: Linking multiple instances via Websocket API~~ Remote Home-Assistant: Linking multiple instances via Websocket API May 19, 2018

MartinHjelmare reviewed May 19, 2018

View reviewed changes

homeassistant added the cla-signed label May 19, 2018

houndci-bot reviewed May 19, 2018

View reviewed changes

Add a stub test

0ca296e

balloob force-pushed the feature_remote_homeassistant branch from f068a5a to 0ca296e Compare May 19, 2018 22:09

homeassistant added the cla-signed label May 19, 2018

balloob closed this May 19, 2018

home-assistant locked as resolved and limited conversation to collaborators Aug 5, 2018


		self.hass_main = get_test_home_assistant()

		#self.hass_main.block_till_done()


		_LOGGER = logging.getLogger(__name__)

		CONF_SLAVES = 'slaves'

Remote Home-Assistant: Linking multiple instances via Websocket API #13876

Remote Home-Assistant: Linking multiple instances via Websocket API #13876

Conversation

lukas-hetzenecker commented Apr 14, 2018 • edited Loading

Description:

Example entry for configuration.yaml (if applicable):

Checklist:

cmsimike commented Apr 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsimike commented May 2, 2018

balloob commented May 2, 2018

lukas-hetzenecker commented May 2, 2018

cmsimike commented May 2, 2018

lukas-hetzenecker commented May 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

balloob commented May 19, 2018

godinperson commented May 23, 2018

lukas-hetzenecker commented May 23, 2018

trebolcinco commented May 27, 2018

steinis123 commented Aug 5, 2018

lukas-hetzenecker commented Apr 14, 2018 •

edited

Loading

Example entry for `configuration.yaml` (if applicable):