-
Notifications
You must be signed in to change notification settings - Fork 212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Processor Boot Concept #62
Comments
I'm a graphical person, so I did the following diagram:
I'm not sure about loading DMEM through the bootloader. How does that work? The bootloader initializes both IMEM and DMEM? Does it initialize IMEM only, and the software app from IMEM then initializes DMEM as part of the "app startup"? |
Oh, one thing I forgot...
Do you mean there are two configurations?
Or four configurations??
I think the two configurations option makes more sense. 🤔 |
I wonder how FPGA-architectures without pre-loadable RAM/ROM fit in the boot concept. For example, we use Microchip PolarFire which has BRAMs that cannot be loaded by bitstream. But it has a NVM, which can be programmed by the bitstream. So we would try to use the NVM to store the program and then try to load the IMEM from the NVM at start-up. Is this currently easily doable? |
The last resort is to use LUTs for implementing the ROM.
Sure. You could write a wrapper for the NVW and replace the default Another option would be to couple the NVM via Wishbone/AXI and rewrite the bootloader to just contain a " |
So, I updated the diagram, taking into account that DMEM is always required and that both IMEM and DMEM can be either internal or external:
I think it is relevant to explain the difference between N/D, I/ID and B/BD, which is the size of the main/only program memory:
In the orange boxes, I mean three options:
It might seem that having a ROM IMEM along with the bootloader might not make sense. However, that can be combined with an external IMEM (E-IMEM). So, at startup, the bootloader would check if a new application is to be loaded. If so, it would load it to E-IMEM, and jump there. Otherwise, it would fall back to the internal IMEM. |
This is what I/ID try to illustrate, in comparison to B/BD. The BootROM implementation is not expected to be customised, but the IMEM implementation is replaceable (as we are doing for radiant and ghdl-yosys-nextpnr). |
I am still trying to figure out some exemplary use cases for your updated memory configs 😅 I just want to emphasize one point again: the CPU does not care where it fetches data or instructions from. This is not a fixed Harvard architecture - it is more like a modified von-Neumann architecture. There is no such thing like an exclusive address space (=memory) for instructions or data. "IMEM" and "DMEM" are just "user" or "software" concepts. So one possible setup can be (assuming customized linker script, bootloader, etc.):
Good point. 🤔 I know there are a lot of different concepts for defining a boot process. But maybe the processor setup does not have to provide pre-defined options for all of them. I think it would be sufficient to provide something like "the most common" ones (I know, this is relative):
|
The point is that I am not proposing new use cases to be supported, but just trying to understand/describe what's possible already 😆 . I agree that some of those are not the most expected, and we don't need to explicitly explain more than 2-3 in the user guide. However, I think there should be some place where the flexibility is made explicit.
Understood. Yet, my concern is the hardware point of view. As you can see in the diagrams, I care about how is each of the memories described in hardware, how is it connected, and how is it initialised. I'm a hardware guy 😄. So, even if IMEM and DMEM can be the same for the software, they are different for me from the hardware point of view. For instance, I can load the main app to the BootROM and replace IMEM with a custom peripheral, so that I have an additional directly accessible interface. Being von-Neumann, the CPU can work with a single memory which is used as IMEM and DMEM at the same time, isn't it?
From a software point of view, that is exactly equivalent to having internal IMEM and no internal DMEM, isn't it? The difference in hardware is that IMEM can be a ROM and DMEM cannot.
In fact, I don't think the processor setup needs to have pre-defined options for any of them. I'm good with having a table such as the one above in the documentation, where we highlight the most common processes. Then, for each board, we would select the solution that best fits. For instance, on the Fomu, the best solution is very different if an USB-to-UART component is available, or if the external SPI is used, compared to not using/having any of those resources.
These do not cover the use case that brought us here 😄: using the bootloader for the (up to 32KB) main/only app; without IMEM (neither internal nor external).
I think this is an interesting feature in any case. |
Absolutely! 👍
I just wanted to underline the fact that boot process is completely up to the (mercy of the) user.
Correct.
Oh, yes, you are right.
Yeah but no... I would like to remove the "initialized IMEM as RAM" concept, since it does not make any sense to me. Btw, I am still experimenting with memories, but these seems to work:
|
due to boot cofniguration issues #62
How about this: Memory SetupThere are three internal memories right now:
I would like to constrain these internal memory types:
Using only internal memories there would be two boot configurations:
Different "memories" could still be used to customize the boot process: Either by replacing the default Boot CondigurationRemove
🤔 |
Gotcha. That was actually something I learnt in this issue 😉. I mean, I knew it was open, but not as flexible as it is!
Oh, I didn't want to imply you are a software guy 😆 but that I feel very unconfident in that area...
I think this is very interesting to be explicitly explained in the docs. It was obvious to me that separated I and D mems could allow having a ROM IMEM and a RAM DMEM. But it was not obvious to me this "defensive" application by introducing a dead address space.
In theory... but theory and practice do not always match. I found three different results on Fomu while trying the same practical solution: a single ROM initialised with the application software (not the bootloader software). One of the solutions required 97% of the resources, the other 51% and the last one 49%. The difference between 51% and 49% was using NoBootROM+IMEMROM or BootROM+NOIMEM. So, even though both of those solutions are exactly equivalent from a software point of view, there is some non negligible difference in the hardware that is generated. Hence, if we can get the IMEM ROM only design to be equivalent to the Boot ROM only design (i.e. 49%), I'm ok with not supporting the latter. We need to understand where is that difference coming from, though.
As commented later, IMAM as RAM is a requierement if the bootloader is used.
My understanding is that you want both IMEM and DMEM to be always required. Then, whether IMEM is a ROM or a RAM is defined implicitly by the existence of the bootloader. I think that is ok. My point of view was complementary: make DMEM required only, and make IMEM an optional RAM only. However, I understand that my approach would constraint the maximum size of the software, while yours does not. Again, I'm only concerned with the size difference. Maybe that is because BootROM is limited to 32KB and IMEM can be up to 2GB?
Note that this is not possible on all FPGAs. We need to take into account use cases where the bootloader is, in fact, an RTL module that will read data from any external interface and preload the IMEM with that. From a software point of view, this is the same as not having a BootROM and the IMEM being a ROM. However, from a hardware point of view, there is component which needs access to the internal bus, for writing into the memory. This is actually a use case I know some users of NEORV32 do have, or will have in the near future. Not something we need to solve now, but we should not forget about it.
The idiomatic solution is a custom enumeration type. The neorv32 library is already an indirect dependency of any synthesis/simulation implying the NEORV32. I think it is fair to require it for instantiating the templates or the top. I'm not sure about implementing the BOOT_CONFIG in the templates or in the top, though. If done in the templates, that would imply some duplication, but it would not modify the codebase of the CPU/SoC. The duplication can be handled through helper functions in a neorv32.boot_config package. I guess you know the complexity better for evaluating that. Furthermore, the BOOT_CONFIG might be not only an enumeration, but a record with several fields. Apart from using the bootloader or not, I see it might be interesting to specify the default boot address, the origin and/or destination addressed for the bootloader, the size of the firmware, etc. Again, this additional metadata is already provided in some other way. Furthermore, all of those fields might be overriden in software. Therefore, it's again up to you, the expert. I'm just thinking loud here. |
👍 I will add a note to the documentation.
This is interesting. Maybe it's a thing the bootROM being implement as arrays of 32-bit while the current IMEM ROM is implemented as four arrays of 8-bit. 🤔
Good point! I think I will update the IMEM, DMEM and BOOTROM memories. I have already tested some new approaches and these seem to be the best ones:
Right. But I was pointing at the "initialized" thing. IMEM will be RAM, but it will be "blank" RAM.
That should not be a problem. A user could use a simplified version of the bootloader that - for example - just loads some exe from an external SPI flash. This could be done with ~64 bytes (optimistic) of bootloader code. I think that every toolchain - regardless of the actual FPGA architecture - should be able to build a LUT-only ROM from this. 🤔
Ok, then this approach wins 😄
I think I would prefer this directly for the top as the ROM option for the IMEM would be redundant. The BOOT_CONFIG could be hidden in the setup-specific wrappers.
I agree. But I think it is a good start to begin with just the two aforementioned boot configuration. If they prove to be practical we can add further customization options. |
I thought about that, but it cannot be that only. The byte enable is 4 gates only. Say there is a 4*1024 byte memory. That can be implemented through eight 512x8 BRAMs. Having 4 gates should be absolutely negligible, because the byte enables are evaluated before routing the wr_en or rd_en signal to the BRAM. However, the address might be relevant. If the width of the address of the IMEM ROM is larger that the one of the BootROM, that size is not optimised, all the bits are routed to the BRAM. Nonetheles, by looking at the sources of the memories, the size of those addresses is extracted from the generics. Need to look deeper into it...
Those look nice 😄 Looking forward to testing them. You might want to describe the ram_t as a unconstrained array of a constrained type (std_logic_vector(31 downto 0)). That should allow to write the init function once only, instead of duplicating the type and function in all the sources.
👍🏼
My point is that the software bootloader does not need hardware access to the IMEM/bus. The BootROM is an slave. The CPU reads the code from that slave, executes it and uses the other peripherals for getting data and writing it to the IMEM. However, a hardware bootloader needs to have master access to the IMEM/bus, because the CPU cannot fetch the software from any pre-initialised memory. Having a LUT-only ROM is an option, but that would only work if the bootloader is short enough and if it can be used with the built-in peripherals (SPI, UART). If data is to be loaded from some unique interface (not supported as a CPU peripheral), the CPU is useless until the firmware is in place. Naturally, you could argue that this is not to be a suppoert use case, and an external IMEM should be used for this.
Good point.
Absolutely. |
currently, this cannot be covered by the riscv-arch-test due to boot concept issues #62
I know that you can have a look at something like an RTL viewer, right? But is there something like a technology viewer to see actual mapping results? 🤔
Good point! I have included that.
Right. You need software to load software 😄 When you use a boot concept that does not use the internal bootloader and fetches an executable from outside of the processor it is up to the system designer to provide a mechanism to initialize the according memory. I have tried to address this in two new sections of the datasheet:
I still need to write something regarding this for the user guide. This whole boot concept is pretty complex as there are dozens of different approaches that might fit a very specific setup. I think it as sufficient (for now) to directly support the most common scenarios and to give some information for implementation more application-tailored setups. |
I'd say it's the other way. It's easier to get a technology viewer (or an RTL that is so close to being a technology viewer), rather than an RTL view closed to the HDL description. See ghdl/ghdl#1519 (comment). I used yosys and netlistsvg in ghdl/ghdl#1783. There are different options you can use, for getting different levels of details and/or symbols:
I really like the updated docs (https://stnolting.github.io/neorv32/#_memory_configuration and https://stnolting.github.io/neorv32/#_boot_configuration)! Neither too simple nor too complex, |
Looks nice! I need to check that out.
Thank your very much! I really appreciate that. I need to do some more fine tuning their, but I think the general concept(s) should be clear now 😉 |
The boot concept(s) seem to be clear now so I think this can be closed. |
This is a "design note" inspired by #56 (especially #56 (comment) and #56 (comment)).
Currently, the processor provides 3 boot options:
BOOTLOADER_EN => true
) to upload executables via UART or from external SPI flash. In this scenario the internal IMEM is implemented as UNinitialized RAM.BOOTLOADER_EN => true
). IMEM is implemented as initialized RAM (initialized with actual executable). The content of the RAM can be modified by software as it is a RAM.BOOTLOADER_EN => true
). IMEM is implemented as initialized ROM (MEM_INT_IMEM_ROM => true
, initialized with actual executable).I do not see any practical use case for option 2, so I think this could be removed?! 🤔
If option 2 is not needed anymore, the boot configuration could be simplified by replacing
BOOTLOADER_EN
andMEM_INT_IMEM_ROM
generics by a single boot config generic (what name? what type?).The text was updated successfully, but these errors were encountered: