See if we can fit WizNet net support into Fuzix on Spectrum +3 and similar

[EtchedPixels]

This may require using the WizNet paged memory space somehow or DivIDE space in order to fit it into a 128K system, particularly a 128/+2.

[EtchedPixels]

Initial sizing suggests that on a +3 it would need the net_w5100.c code rewriting to be standalone and banked in the WizNet paged memory space. Probably also in asm therefore due to the separate banking and the need to run that code with interrupts off (because the Spectranet has a brain dead assumption it can provide a fixed low replacement ROM area and appears not to vector it via RAM)

On a 128K banked machine the same would apply. On a 256K+ system it ought to fit just fine without this but given it will be needed on the others it makes little sense not to.

[EtchedPixels]

net_w5100 is about 2K, and would end up mapped low so only cause banking problems on the +3 where a copy between user and w5100 could be hard without a bounce buffer. On the 16K window systems only the top 32K is user space, on the TC2068 we have to keep the low chunk for kernel because of the memory layout.

[pawosm-arm]

As I wrote in the other ticket: I've got Spectranet device attached to my +3e machine, yet it has its firmware jumper-disabled (as it makes it impossible to boot any OS from +3 Loader menu), so currently, the device itself is left unused.

Similar setup (Spectranet with firmware disabled) seems to be possible with FUSE emulator:

...yet I was never able to test if either FUSE or the real thing behaves as Ben Versteeg reassured me:

Spectranet, DivMMC or DivIDE can have their firmware disabled, and indeed the hardware is still available.

Some simple code, verifying behavioral and compatibility assumptions (either in Fuzix kernel, or bare-metal assembly), would be a good starting point.

[pawosm-arm]

Looks doable providing the spectranet handles the +3 memory map correctly (ie it pages in over the ROM if the ROM low memory range is triggered, not just if you access 3E00-3FFF in CP/M memory modes)

I suspect we're running into some misleading assumptions here. As the user guide states, the 'disable' jumper's role is "to disable ROM" (effectively, this prevents Spectranet from forcing any 128k model of ZX Spectrum to always start in '48k BASIC' mode), and, according to the guide, it is 'required for firmware updates'. I can imagine, that accessing trapped addresses has no effects in that mode. It seems that the following manufacturer recommendation must be taken in reverse:

Avoid using OUT instructions (especially while the Spectranet is in the prototype stage) to directly perform Spectranet operations unless there's a really, really good reason. If you just want to cause Spectranet memory to page in without actually running a Spectranet function (for example, to access the additional memory), use CALL 0x3FF9 instead.

Hence, I assume that by 'direct talking to hardware when firmware is disabled' they specifically meant IN/OUT instructions that they recommended not to use in typical scenarios and that this is the only way to make things happening when firmware is disabled...

[pawosm-arm]

When Spectranet emulation is enabled along with Spectranet ROM disabled, +3 BASIC's PRINT IN 827 prints current border value (values range 0-7) on FUSE emulator. It prints border value AND 64 (values range 64-71) on the real thing (so the bit marked as W5100 interrupt state is set on the real thing, while it is reset on FUSE; lowest 3 bit are filled with current border color). When Spectranet emulation is disabled in FUSE (or Spectranet device not attached to the real thing), +3 BASIC's PRINT IN 827 always prints 255.

Doing the same on the real thing in 48k BASIC having Spectranet ROM enabled prints border value AND 128 (values range 128-135, so the highest bit, marked as reserved, is set, while W5100 interrupt state bit is reset).

In theory, OUT 827, 1 should page-in Spectranet memory even with Spectranet ROM jumper-disabled. Doing it in +3 BASIC causes crash (rather expected), although less spectacular on the real thing (it just freezes) than on FUSE (strips are displayed on screen).

[pawosm-arm]

To move further, I had to depart from safe shores of ZX BASIC and go straight to the assembly. So I did some experiments using my own small OS. And I was lucky that I dedicated Bank 2 for programs that talk to the hardware.

First observation: FUSE always handles Spectranet paging for any 128k/+3 memory layout. Which is huge advantage, but we are targeting the real thing rather than emulation.

My test program was running in Bank 2 within special (+2A/+3) layout 0. Whole paging and testing read/write access was done between DI/EI instructions. Writing to port 827 (0x033B) has no effect, I had kernel in Bank 0, and it was still there (verified first two bytes known to me). After turning to 128k memory layout, Bank 2 was still in the same place, so my program could continue and page-in Spectranet to the address space of Bank 0. This area is divided into four 4kB pages. First page is always Spectranet ROM (I checked it, read value, display, wrote value, read again in the second pass, display), two next pages can be selected by writing their Spectranet numbers to two ports: 0x003B (first page number) and 0x013B (second page number). I've selected various pages, some of them were internal RAM (pages number 0xC0-0xDF), some of them were unused ROM addresses (0x04-0x1F), and indeed, their value didn't change during compare test (as described earlier). One side note: data written to RAM survive reset! It takes complete power down to get rid of them.
Requesting page-out by writting to port 827 (0x033B), then returning to +3 mode special layout 0 and enabling interrupts resumes normal OS life.

The code I tried was:

die_hard:
  ld a, #0b010
  out (#0xfe), a
  halt

verify_kernel:
  ld a, (#0x0000)
  cp #0xc9
  jr nz, die_hard
  ld a, (#0x0001)
  cp #0xff
  jr nz, die_hard
  ret

verify_p3dos:
  ld a, (#0x0000)
  cp #0x18
  jr nz, die_hard
  ld a, (#0x0001)
  cp #0xfe
  jr nz, die_hard
  ret

verify_not_kernel:
  ld a, (#0x0000)
  cp #0xc9
  ret nz
  ld a, (#0x0001)
  cp #0xc9
  ret nz
  jr die_hard

verify_not_p3dos:
  ld a, (#0x0000)
  cp #0x18
  ret nz
  ld a, (#0x0001)
  cp #0xfe
  ret nz
  jr die_hard

  .globl _spcnet_asm_part
_spcnet_asm_part::
  pop iy
  di
  xor a
  out (#0xfe), a
  ; currently, we're working with the +2A memory model, special layout 0
  call verify_kernel
  call verify_not_p3dos
  ; switching to the 128k memory model
  ; keep in mind that the bootloader has sent 0 to the port 0x7ffd
  ; and double buffering routine has been flipping bit 3 of that port
  ld a, #0b100 ; let's have +3DOS in ROM
  ld bc, #0x1ffd ; +3 register
  out (c), a
  call verify_not_kernel
  call verify_p3dos
  ld bc, #0x033b ; cpld register
  in a, (c)
  and #0b00101000 ; remember relevant bits of the cpld register state
  ld l, a
  ld a, #0x04 ; first Spectranet Flash ROM unused page
  ld bc, #0x003b ; paging area A @0x1000
  out (c), a
  ld a, #0xc0 ; first Spectranet 4kB RAM page
  ld bc, #0x013b ; paging area B @0x2000
  out (c), a
  ld a, l
  and #0b11110111 ; disable programmable trap
  or  #0b00000001 ; page-in
  ld bc, #0x033b ; cpld register
  out (c), a
  ; now we're paged-in
  ld a, #0b111 ; let's blink with white border for a jiffy
  out (#0xfe), a
  call verify_not_kernel
  call verify_not_p3dos
  pop bc
  ld a, (bc)
  push af
  inc a
  ld (bc), a
  pop af
  push bc
  ld bc, #0x033b ; cpld register
  out (c), l ; page out
  ld l, a
  call verify_not_kernel
  call verify_p3dos
  ; switching back to the +2A memory model, special layout 0
  ld a, #0b001
  ld bc, #0x1ffd
  out (c), a
  call verify_kernel
  call verify_not_p3dos
  ei
  halt ; keep the border white until the next IRQ from ULA (kernel changes it)
  push iy
  ret

It was called from the _start() function of this process as such:

#define EXAMINE(coid, where) (putbits((coid), spcnet_asm_part((void *)(where))))

  puts(conid, "Welcome to the Spectranet experiments.\n");
  putbits(conid, *((uint8_t *)(0x1000U)));
  putbits(conid, *((uint8_t *)(0x2000U)));
  puts(conid, "\n");
  /* first pass */
  EXAMINE(conid, 0x0000U);
  EXAMINE(conid, 0x0001U);
  EXAMINE(conid, 0x1000U);
  EXAMINE(conid, 0x2000U);
  puts(conid, "\n");
  /* second pass */
  EXAMINE(conid, 0x0000U);
  EXAMINE(conid, 0x0001U);
  EXAMINE(conid, 0x1000U);
  EXAMINE(conid, 0x2000U);
  puts(conid, "\n");
  putbits(conid, *((uint8_t *)(0x1000U)));
  putbits(conid, *((uint8_t *)(0x2000U)));
  puts(conid, "Experiment finished.\n");

The effect is visible on the attached screenshot.

[pawosm-arm]

This experiment inspired me to create FID driver for CP/M. It enables use of on-board extra RAM as a ramdisk. Can be downloaded from here: https://sourceforge.net/projects/spectramfid

[pawosm-arm]

Some news related to this one. I was corrected by @ZXGuesser regarding assumed inability to boot Fuzix with all of the Spectranet's on-board jumpers set to 'enable'. Although fully enabled Spectranet device forces +3 machine to boot into 48k (USR 0) mode, it is possible to reset into +3 power up menu and boot Fuzix (or CP/M Plus, or any other 128k software) with following short piece of code:

xor a
ld bc,0x7ffd
out (c),a
ld b,0x1f
out (c),a
jp 0x0000

...which can be expressed in 48k ZX BASIC as such:

10 DATA 175, 1, 253, 127, 237, 121, 6, 31, 237, 121, 195, 0, 0
20 RESTORE 10
30 FOR i = 0 TO 12
40 READ a
50 POKE 23296 + i, a
60 NEXT i

RUN
GOTO USR 23296

I wonder how much easier development of the network driver would be with this. I'll try to craft something simplistic for CP/M Plus (where at least I know some of its FID drivers API), yet I don't know any time frame for doing so yet.

[pawosm-arm]

The boot hack above is not needed. I finally managed to prepare CP/M's FID driver that initializes Spectranet device which has both of the 'disable' jumpers closed. This driver (spectran) exposes a character device (allowing serial, console or line printer over ethernet) and a block device allowing remote access to a disk image (working with this makes impression of having a LANtastic alike network drive). Also I've prepared a GSX graphics driver (spectrangsx) working on top of my FID driver (by means of CP/M BIOS calls) allowing access to a remote display server working on Linux (somewhat like working with X server over the network). I also made use of the free RAM installed on Spectranet board exposing it as a block device (a ramdisk) with yet another FID driver (spectram). Turned out, the networking code in the firmware requires only the first page of on-board RAM, the rest can be entirely used for general purposes, yet due to functionality limitations of the CP/M's FID drivers (only character or block devices can be registered by a FID), a ramdisk was the only way of exposing this memory to the user.

Yet all of those achievements in CP/M don't seem to make Spectranet device any closer to Fuzix. CP/M was never considered a networking system (e.g. it does not have socket library), and even CP/Net isn't the thing we're expecting to use nowadays. Hence to cut corners, in my driver I used socket library as exposed by the firmware. The Spectranet firmware sources are available on github (https://github.com/spectrumero/spectranet.git), at some point I managed to build it and reflash my Spectranet device with it. All it needed was binutils (with GNU as) configured and built for targeting z80-unknown-coff.

To have Spectranet driver in Fuzix, a completely different approach needs to be taken. Fuzix has its own layers of networking infrastructure with its own socket library being one of those layers. Also, all of the trickery I did in order to escape +3 specific memory layouts, page-in the firmware and call its functions could only work with a system as simple as CP/M (e.g. I could force it to load my FID driver always into Bank 5, right after framebuffer, so it remained under the same addresses as I was switching the memory layouts), I suspect, this should not be done with Fuzix. It is apparent that whatever the firmware does to talk to the hardware must be done by a Fuzix driver. This includes all the buffering that is needed. And since access to the on-board RAM requires all of the trickery with escaping +3 specific memory layouts I suspect this vast reservoir of memory won't be available to Fuzix. In effect, significant chunk of the precious RAM will be taken by the Spectranet driver (for its code and data), increasing the overall swap pressure. It makes me wonder if it's worth to continue with this, we may end up with network-enabled unusable system.

[EtchedPixels]

It may have gotten a bit easier at this point. I've been reworking the network code so that in theory it can run as a completely separate binary with just a few entry/exit points between it and the Fuzix kernel proper, plus a small amount of shared memory space between the two (an 8 byte table and about 64 bytes of udata). The +3 isn't the main reason to do it - there are a bunch of systems where general banking of a binary is not doable or has nasty performance impacts so I borrowed the idea from 2.11BSD which does something similar

[pawosm-arm]

Wow, these changes look impressive. I've got some preliminary work started on supporting the network interface of dk-tm4c129x which seems to be easily accessible, yet it looks like I need to sort out some of my accommodation issues before I resume it. With +3 it seems little scary having in mind all of the trickery I had to impose to make the Spectranet networking device work with CP/M. But then, there is a huge historical gap between those two hardware platforms which results in my entirely bad way of thinking about it.

[EtchedPixels]

Thinking about this I am wondering if it's better to make some tradeoffs and use the non +3 mappings.

If you reduce the max user app size to about 36-40K (which is not too bad IMHO) then a bunch of nice things happen providing you have a SpectraNet

• Kernel data and bss fits between 1000 and 3A00
• Kernel common fits either at 4000 or 5B00 depending when you put the screen
• The 8000-BFFF page starts with a proper discard
• C000-FFFF holds a banked kernel and two user pages we can hardware swap

The lower space for the user process we end up having to copy to/from on a task switch as with the Spectrum 128K and DIVMMC but that is not really any more expensive than writing to/from disk, especially as there should be enough Spectranet RAM to copy to/from to avoid having to implement an expensive "exchange" op.

Networking would then definitely fit if you limited userspace to about 36K and put video at 0x4000, or you could go for a bigger process size and video in the upper space but that might be rather tighter.

The actual spectranet firmware would need some tweaks or one or two routines re-implementing (notably non-blocking connect) but not much. And actually it would be good if that was fixed for spectrum use as well if it's not been addressed in newer firmware.

[pawosm-arm]

Interesting idea. I wonder if the on-device SRAM could also be used by FUZIX, there is a plenty of it and only a small chunk of it is used by the networking routines, leaving the rest for general use (in my CP/M driver, I'm exposing it as a ramdisk, as there is no better way of making use of it in CP/M; hopefully it's more flexible in FUZIX).

I hope that apart from memory layouts we're not losing any other +3 features (e.g., access to the FDD).

Also I hope that apart from DIVMMC, the ZXMMC support is also considered, that's the only hardware setup I do have.

Dealing with the firmware sounds scary (for any potential future user), would be good to avoid it (not to mention backward compatibility with existing software), despite the fact I was rebuilding it from sources and reflashing while investigating some networking issue (in the end it turned out that it was the network switch which didn't like spectranet to blame, so I reverted all of my changes and reflashed original firmware).

[EtchedPixels]

I built most of it to see and even assuming raw W5100 bashing it seems to fit.

See platform-zxspectra

Still a couple of interesting questions to answer, notably how to make map_for_swap and swapping work sanely. However the target has its own bank code and swap logic so if need be it can simply cheat.

[EtchedPixels]

I don't believe there is anything at the hardware level tied to being in a particular mode like that (unlike the betadisk mess). Certainly the floppy drive can be used in 128K mode and is by basic.