Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs/SDK/notebooks with nbdev [try 2] #103

Closed
wants to merge 30 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
17c8f06
init docs/SDK/notebooks with nbdev
brunosan Dec 27, 2023
281288c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 27, 2023
87fc30b
ignore locally rendered output
brunosan Dec 27, 2023
5ec27c8
these files should not be tracked. ^1
brunosan Dec 27, 2023
893f255
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 27, 2023
bb86ffb
For now, I do want to preview documentation coming from this branch
brunosan Dec 27, 2023
ae1cf85
Merge branch 'nbdev2' of github.com:Clay-foundation/model into nbdev2
brunosan Dec 27, 2023
4a1c44b
making ruff happy.
brunosan Dec 27, 2023
3bfad0c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 27, 2023
a7d92fd
lint ruff
brunosan Dec 27, 2023
41dedbf
Merge branch 'nbdev2' of github.com:Clay-foundation/model into nbdev2
brunosan Dec 27, 2023
a190897
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 27, 2023
595c133
better explanation of embeddgins
brunosan Dec 27, 2023
99e7b70
Merge branch 'nbdev2' of github.com:Clay-foundation/model into nbdev2
brunosan Dec 27, 2023
1a4ca16
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 27, 2023
0a1c681
basic rgb read
brunosan Dec 27, 2023
e88bc87
more rebust rgb handling
brunosan Dec 27, 2023
850a13b
better handling of RGBs
brunosan Dec 27, 2023
812046c
exlore semantic peaks
brunosan Dec 27, 2023
ec59edb
nbdev_clean
brunosan Dec 27, 2023
29fde8d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 27, 2023
7a8e553
Need to use python 3.10 for type Union check on function using |
brunosan Dec 27, 2023
733dcf2
Merge branch 'nbdev2' of github.com:Clay-foundation/model into nbdev2
brunosan Dec 27, 2023
c432651
Handle location indices
brunosan Dec 27, 2023
34bacf0
handle bounds with default order
brunosan Dec 27, 2023
8f60256
fix index reading and drop conflating col.
brunosan Dec 27, 2023
2da4e20
beter ims using 10-90 percentiles
brunosan Dec 27, 2023
59c9d02
iloc for loc
brunosan Dec 27, 2023
d5b5fc9
smaller plots
brunosan Dec 27, 2023
c0647f0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/deploy-docs.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Deploy to GitHub Pages
on:
push:
branches: [master, nbdev]
branches: [master, nbdev2]
workflow_dispatch:

permissions:
Expand All @@ -14,7 +14,7 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.9'
python-version: '3.10'
cache: "pip"
cache-dependency-path: settings.ini
- name: Install Dependencies
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,6 @@ docs/_build/
# Mac OS-specific storage files
.DS_Store
datadisk/
docs/_docs/
docs/clay/
docs/_proc/
1 change: 1 addition & 0 deletions docs/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.ipynb merge=nbdev-merge
11 changes: 11 additions & 0 deletions docs/.gitconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Generated by nbdev_install_hooks
#
# If you need to disable this instrumentation do:
# git config --local --unset include.path
#
# To restore:
# git config --local include.path ../.gitconfig
#
[merge "nbdev-merge"]
name = resolve conflicts with nbdev_fix
driver = nbdev_merge %O %A %B %P
62 changes: 62 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Clay documentation

<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

## Overview

Clay is a [foundational model](#) of Earth using Earth Observation data.
As the AI Deep Learning architecture, it uses an expanded [visual
transformer](#) upgraded to understant geospatial and temporal relations
on Earth data, from any instrument/spectral data. The AI self-supervised
fundational task is a [Masked Autoencoder
(MAE)](https://arxiv.org/abs/2111.06377) approach for training.

The Clay model primarily functions in two ways: first, by directly
generating semantic embeddings for tasks like similarity searches, and
second, through fine-tuning its outputs with additional data labels.
This fine-tuning supports various tasks, including classification
(e.g. flood detection and deforestation monitoring), regression
(e.g. estimating carbon stock or crop yields), and generative tasks such
as creating RGB imagery from SAR data. Moreover, users can further
enhance model performance by incorporating higher-resolution data.

This documentation uses [nbdev](#), which combines documentation, code
samples and an SDK. This means that every page is also a python notebook
anyone can use, with practical code examples for each functionality, and
use case. Moreover, you can install `pip install clay` and use the same
functions.

Clay is open source, open data and open for business.

## Where is what

- The Clay model **code** lives on
[Github](https://github.com/Clay-foundation/model). License:
[Apache](https://github.com/Clay-foundation/model/LICENSE).
- The Clay model **weights** live on [Huggin Face](#). License:
[OpenRAIL-M](https://github.com/Clay-foundation/model/blob/main/LICENSE-MODEL.md).
- The Clay **documentation** lives on here. License: [CC-BY](#).
- The Clay **SDK** lives on [PyPi](#). License: [Apache](#).
- We maintain a set of **embeddings** on [Source Cooperative](#).
License: [ODC-BY](#).

## How to use Clay

The model can be used in two main ways:

1. Directly, use it to make inference. See [Model](Model.html)
1. Check and run Benchmarks on the model. See
[Benchmarks](Benchmarks.html)
2. Generating semantic **embeddings**. E.g. for Similarity search. See
[Embeddings](Embeddings.html).
3. **Fine-tunning** the model for other tasks, or for other input data.
E.g. flood detection, crop yields, … See
[Fine-tunning](Fine-tunning.html).
4. **Generative** tasks: E.g. Estimate RGB imagery wihtout clouds or
from Synthetic Aperture Radar (SAR) data. See
[Generative](Generative.html).


Clay is a fiscally sponsored project of [Radiant Earth](), a USA
registered 501(c)3 non-profit.
48 changes: 0 additions & 48 deletions docs/_config.yml

This file was deleted.

26 changes: 0 additions & 26 deletions docs/_toc.yml

This file was deleted.

1 change: 1 addition & 0 deletions docs/clay/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__version__ = "0.0.1"
9 changes: 9 additions & 0 deletions docs/clay/core.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/00_core.ipynb.

# %% auto 0
__all__ = ["foo"]


# %% ../nbs/00_core.ipynb 3
def foo():
pass
57 changes: 0 additions & 57 deletions docs/data_labels.md

This file was deleted.

5 changes: 0 additions & 5 deletions docs/installation.md

This file was deleted.

13 changes: 0 additions & 13 deletions docs/intro.md

This file was deleted.

Binary file removed docs/logo.png
Binary file not shown.
Loading