@nemesisdesign please can you give me directives on how to reproduce this issue on my local repo? so that i can better visualize the issue in other to easily get a fix for it. thanks

Okay, so just to confirm if I have understood it correctly.
I need to add a flag say --new-registered that will be passed when a new device is registered. This will make sure that if the key exist in template, then it's used instead of the one received from device.

However, if --new-registered is not passed the behavior does not change (because currently config is already given precedence over templates.)

Am I correct in my understanding of the issue? 😄

Kind of a flag, I'd think it more as a function argument.

I am not sure you understood the aim of this change.

We need to make able OpenWISP to capture the configuration from the devices, so we'll send the full config from the devices to OpenWISP Controller which in turn will call netjsonconfig to understand what to do.

Let's start with the more common case, normal updates.
In normal updates, we just have to be sure that the part of the config coming from the device which is equal what is present in templates doesn't get duplicated in the config of the device.
We build a netjsonconfig backend instance passing config and templates, then we call a method, this method will update the instance so that the config sent by the device is included in the object, this config being separated in config and templates so that the part of the config that is the same to the templates is not duplicated in config. Then we will save this in OpenWISP Controller.

The other case is when a device registers. I'm not 100% sure about the solution proposed in this old ticket. I would discuss that further before proceeding.

BTW, until this point, have you understood something?

BTW, until this point, have you understood something?

I think I am able to understand the problem.
Now, About how I have implemented the part sending device config to server is called after openwisp-controller configurations are applied to the device.
Hence, the case 1 mentioned in the issue will not happen, instead, only case 2 happens.

As of case 2, please correct me if I have misunderstood anything about what we want:

1. Receive full config from device
2. Compare the config and template saved in the controller with received config
3. Find keys with different values and store these values in config of the device.

** Note: I am assuming that we convert the configs we received from device to NetJson DeviceConfiguration json and we compare these keys to the keys of NetJson DeviceConfiguration stored in the controller.

So, bottom line, we need a function to compare two different DeviceConfigurations and return keys that have different value and store those values.

Is my understanding correct?

@atb00ker: I'll try to explain with some pseudocode.

We need to be able to do the following:

backend_instance = OpenWrt(
    native=tar_gz_fileobj,
    config=device.config,
    templates=[management_vpn, snmpd]
)

management_vpn and snmpd represent default templates that are stored in OpenWISP.

The result should be a backend_instance which will get:

• all the configuration that is passed to native (the config received from the device), and is also available in templates, will not be present in backend_instance.config
• the rest will be put in backend_instance.config

At that point in openwisp-controller we'll just have to do:

device.config.config = backend_instance.config
device.config.full_clean()
device.config.save()

Is it clearer now?

@atb00ker
So far I have been dealing with the openwisp agent in the openwrt router.
Now I wanted to try the new approach of having the router push the configuration to the server.
I have also adapted the server with this changes in this pullrequest.
So far so good.

But I don't think that's all, is it?

I need more input on what I need to do now
I don't know much about the server.
I need more input from your side.

When you say you have taken changes from pull request, I assume you are talking about openwisp/openwisp-config#107

And yes, that's not enough, after this we would need to complete this work in netjsonconfig #152

When that is complete, the next step is taking that release of netjsonconfig to openwisp-controller (basically update the dependency) and create endpoints to access it from API.
Then we only need to release it in ansible-openwisp2 to complete it.

The blocker from my end & Federico's end is bandwidth, we are not able to get to it, so if you have experience with Django, please test it and see if there are any bugs / problems, if yes, please help us solve them.

That's a high level idea of what remains in this. (I think it is a lot of work considering you don't have knowledge about the server.)
However, if you are willing to take this task, then I'll try to help you as much as I can! 😄