Mapping water tracks in the Arctic!

[jmdelvecchio]

Hi Dan and Evan! Thank you so much for giving the world Doodler and Segmentation Zoo. Dan, I was working along with MLMondays in the fall, and to my delight this week I discovered this project. For my postdoc my goal is to automatically detect Arctic flowpaths known as water tracks and map their networks. After working with your example data (again, thank you for walking us through the process and setting up some pretty painless programs) I chopped up my PlanetScope imagery from western Alaska and I jumped right in!

I scribbled on a mere 18 images from my ~250 total for a June 2019 scene in western Alaska that included classes for snow, shrubs, "streaks" (what I'm calling not-quite-shrubby water tracks), rock, "background" (any non-textured tundra), and nodata. I used the "vanilla" settings and I'm so impressed with the results, considering I still don't know how to tune these models:

Like I said, I'm really new, but this is a great start. Open to suggestions, I know that accuracy is pretty low but to me it's amazing😂. Doodler in particular will be awesome for my student researchers.

I noticed that you're in the process of describing the satunet in the wiki, and I want to know more about that! I messed around with that (I had to edit the .py scripts in the src folder to "simple_satunet" instead of "custom_satunet" to get it to run, was that your intention? I can make an issue/pull request), but I still only gave it my 3 RGB bands and the results were definitely wonky. I know for a fact that both spectral indices and topographic metrics would vastly improve the model prediction, so I'm eagerly awaiting instructions/examples for 4+ band inputs. With Planet's Superdove 8-band images and ArcticDEM-derived topo metrics as inputs I think this model can work out really well.

[ebgoldstein]

oh wow, this is rad. Thank you so much for this message @jmdelvecchio - brightened my day for sure.. And i love seeing output like this.
Satunet- I know from my perspective, I haven;t spent as much time using/testing the sat_unet, especially after a flurry of work in late 2021.. so if you think there is a 🐞, feel free to make a PR or just an issue and someone from ZooHQ (😆) will look into it.

I have seen (recent) results from @dbuscombe-usgs using 4+ band imagery. i think its just having a another folder of 4th band images (in addition to image+label dirs). the make_ND_datasets.py GUI should prompt you for additional stuff after the images+labels, and that is the place where you direct it to the extra bands. it should work out of the box. (:crossed_fingers: )

i also want to say that you might have some success 'remapping' the classes via the config and collapsing the problem to a binary one - 'streaks' and 'nostreaks'. especially in the limit of small datasets, this has been helping us alot.

(but more labeled data is always worth the most in terms of increasing model performance. for 19 images, i think your curve looks pretty rad!!) Keep us posted!!!

[dbuscombe-usgs]

@jmdelvecchio this makes me v v happy, thanks for the message - you rock!

We should better expose and properly document the 'custom satellite' unet. Initial trials would suggest that it is inferior to both the res-unet and unet implementations, but only on limited RGB-only datasets. So, I would encourage experimentation! (I don't think we've tested that model on >3 band images, but it's now on the to-do list)

As @ebgoldstein said, make_nd_datasets should be set up for merging bands to create >3 band model inputs. You are first asked for the label images, then the RGB images, then you have the option of adding bands. In your case, it sounds like you would add a 4th band containing DEM. This should be a regular 8-bit jpeg, remapping your elevation range to the range 0-255. This may be tricky if you have a lot of relief, but please post issues here and I'd be happy to work with you to iron out any kinks. Personally, I have made models for 4,5,and 6 bands and things have worked out (L8 and S2 satellite imagery only so far)

I would second what @ebgoldstein suggested about playing with binary classes. Use the REMAP_CLASSES functionality in the config file. Note that if you use 'remap classes', the NCLASSES parameter in the config file should reflect the number of remapped classes. If you have variable image size, the program will pad the imagery to ensure consistency, but that creates a null or zero-class that should be counted in NCLASSES (i.e. in that specific situation, NCLASSES = NCLASSES+1

And yes, more data should really help the model generalize. Glad you got Doodler working for you! hyperparameter choices (such as batch size, learning rate, kernel size, and number of filters) can strongly improve results on small* datasets, but it is our experience that greater improvement is usually best achieved by labeling more, especially relatively 'difficult' images

Keep us posted! We'll happily (and gratefully) add any clarifying details to the docs, too, just let us know using an Issue

[jmdelvecchio]

Hey fam, just dropping in to report after a term of teaching I'm ready to return to this project, this time with a student who is paid to do all the clicking!! I'm going to start her on just Doodling the Planet RGB imagery while I hack away at adding bands. I have to figure out how to keep the pixels straight between different resolution datasets - 3 m Planet means upsampling 10 m S2 and downsampling 2 m ArcticDEM, right? I assume I can't mix pixel sizes between bands.

[dbuscombe-usgs]

Hi, thanks for the update. Sounds like a fine plan! Yes, finding a common pixel size would be necessary using resampling. I imagine nearest neighbour resampling would work well

Once you have a trained Gym model that you'd like to apply, I have codes that may help with the "geospatial" part .... I'm working up to a blog post or two. I'll ping you when they are completed

[ebgoldstein]

hi @jmdelvecchio,
We do not have common scripts in doodler or Gym for this workflow, but it seems each of us has scripts floating around.. I think the hope was to one day include this info in one (all?) of the doodleverse repository wikis... But for now, from an earlier discussion via email with other Doodleverse users, here are two paths forward.

The first relies purely on shell scripts:

1. When using gdal to convert from geotiff to jpeg for doodling, use the world file output flag. That creates an .aux.xml file (and wld) file per image.

2. When I use gen_images script from doodler to make my ‘predseg.png’ outputs, copy the xml files into the same directory and use gdal with the `predseg' files to reattach the geographic info.

Second option is pythonic, using rioxarray/rasterio.. this still relies on having the gdal outputs.. (i don't have psuedocode for that hanging around)..

[ebgoldstein]

all of this being said - if you want to contribute some code/guidance/documentation, we would be thrilled to take it.

[jmdelvecchio]

Cool, I wanted to make sure I wasn't reinventing the wheel - I'll make you a deal, I'll figure out how to do it and I'll post the whole workflow for the repo. My student has already written a detailed workflow for going from Earth Engine ROIs to Planet imagery to Doodle-able images, and I'll round it out with the geospatial element sometime in the next month or two. Guest blogger?

[ebgoldstein]

great - keep us posted!

[ebgoldstein]

just want to chime in here that this is enough of an issue that Doodleverse HQ is starting to work on this in earnest: https://github.com/Doodleverse/s2m_engine

@jmdelvecchio feel free to pitch in to the effort !!