remove hardcore path and TEMP PRINT lines

Signed-off-by: hsj576 <sjhu21@m.fudan.edu.cn>

examples/robot/lifelong_learning_bench/README.md

@@ -1,26 +1,26 @@
 # Quick Start
 
-Welcome to Ianvs! Ianvs aims to test the performance of distributed synergy AI solutions following recognized standards, 
-in order to facilitate more efficient and effective development. Quick start helps you to test your algorithm on Ianvs 
-with a simple example of semantic segmentation based on lifelong learning. You can reduce manual procedures to just a few steps so that you can build and start your distributed synergy AI solution development within minutes. 
+Welcome to Ianvs! Ianvs aims to test the performance of distributed synergy AI solutions following recognized standards,
+in order to facilitate more efficient and effective development. Quick start helps you to test your algorithm on Ianvs
+with a simple example of semantic segmentation based on lifelong learning. You can reduce manual procedures to just a few steps so that you can build and start your distributed synergy AI solution development within minutes.
+
+Before using Ianvs, you might want to have the device ready:
 
-Before using Ianvs, you might want to have the device ready: 
 - One machine is all you need, i.e., a laptop or a virtual machine is sufficient and a cluster is not necessary
 - 2 CPUs or more
 - 4GB+ free memory, depends on algorithm and simulation setting
 - 10GB+ free disk space
 - Internet connection for GitHub and pip, etc
 - Python 3.6+ installed
-
 
-In this example, we are using the Linux platform with Python 3.9. If you are using Windows, most steps should still apply but a few like commands and package requirements might be different. 
+In this example, we are using the Linux platform with Python 3.9. If you are using Windows, most steps should still apply but a few like commands and package requirements might be different.
 
 ## Step 1. Ianvs Preparation
 
 First, we download the code of Ianvs. Assuming that we are using `/ianvs` as workspace, Ianvs can be cloned with `Git`
 as:
 
-``` shell
+```shell
 mkdir /ianvs
 cd /ianvs #One might use another path preferred
 
@@ -29,9 +29,9 @@ cd project
 git clone https://github.com/kubeedge/ianvs.git   
 ```
 
+Then, we install third-party dependencies for ianvs.
 
-Then, we install third-party dependencies for ianvs. 
-``` shell
+```shell
 sudo apt-get update
 sudo apt-get install libgl1-mesa-glx -y
 python -m pip install --upgrade pip
@@ -41,8 +41,9 @@ python -m pip install ./examples/resources/third_party/*
 python -m pip install -r requirements.txt
 ```
 
-We are now ready to install Ianvs. 
-``` shell
+We are now ready to install Ianvs.
+
+```shell
 python setup.py install  
 ```
 
@@ -51,7 +52,7 @@ python setup.py install
 Datasets and models can be large. To avoid over-size projects in the Github repository of Ianvs, the Ianvs code base does
 not include origin datasets. Then developers do not need to download non-necessary datasets for a quick start.
 
-``` shell
+```shell
 mkdir /data
 cd /data
 mkdir datasets
@@ -64,9 +65,9 @@ we have done that for you and the interested readers can refer to [testenv.yaml]
 
 <!-- Please put the downloaded dataset on the above dataset path, e.g., `/ianvs/dataset`. One can transfer the dataset to the path, e.g., on a remote Linux system using [XFTP].  -->
 
+Related algorithm is also ready in this quick start.
 
-Related algorithm is also ready in this quick start. 
-``` shell
+```shell
 export PYTHONPATH=$PYTHONPATH:/ianvs/project/ianvs/examples/robot/lifelong_learning_bench/testalgorithms/rfnet/RFNet
 ```
 
@@ -79,52 +80,53 @@ If you want to run the large vision model based cloud-edge collaboration process
 
 In this example, we use [SAM model](https://segment-anything.com/) as the cloud large vision model. So, we need to install SAM by the following instructions:
 
-~~~bash
+```bash
 cd /ianvs/project
 git clone https://github.com/facebookresearch/segment-anything.git
 cd segment-anything 
 python -m pip install -e .
-~~~
+```
 
 Then, we need to download the pretrained SAM model:
 
-~~~bash
+```bash
 wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
-~~~
+```
 
 In order to save the inference result, we need to install mmcv and mmdetection by the following instructions:
 
-~~~bash
+```bash
 python -m pip install https://download.openmmlab.com/mmcv/dist/cu118/torch2.0.0/mmcv-2.0.0-cp39-cp39-manylinux1_x86_64.whl
 cd /ianvs/project
 git clone https://github.com/hsj576/mmdetection.git
 cd mmdetection
 python -m pip install -v -e .
-~~~
+```
 
 In case that your computer couldn't run SAM model, we prepare a cache for all the SAM inference results in Cloud-Robotics dataset. You could download the cache from [this link](https://pan.baidu.com/s/1oGGBa8TjZn0ccbznQsl48g?pwd=wpp1) and put the cache file in "/ianvs/project/":
 
-~~~bash
+```bash
 cp cache.pickle /ianvs/project
-~~~
+```
 
  By using the cache, you could simulate the edge-cloud joint inference without installing SAM model.
 
 Besides that, we also provided you a pretrained RFNet model in [this link](https://pan.baidu.com/s/1h8JnUgr1hfx5QnaFLLkMAg?pwd=jts4), you could use it if you don't want to train the RFNet model from zero. This instruction is optional:
 
-~~~bash
+```bash
 cd /ianvs/project
 mkdir pretrain
 cp pretrain_model.pth /ianvs/project/pretrain
-~~~
+in /ianvs/project/ianvs/examples/robot/lifelong_learning_bench/testalgorithms/rfnet/RFNet/utils/args.py set self.resume = '/ianvs/project/pretrain/pretrain_model.pth'
+```
 
 ## Step 3. Ianvs Execution and Presentation
 
-We are now ready to run the ianvs for benchmarking. 
+We are now ready to run the ianvs for benchmarking.
 
-To run the basic lifelong learning process: 
+To run the basic lifelong learning process:
 
-``` shell
+```shell
 cd /ianvs/project/ianvs
 ianvs -f examples/robot/lifelong_learning_bench/benchmarkingjob-simple.yaml
 ```
@@ -134,16 +136,15 @@ e.g. `/ianvs/lifelong_learning_bench/workspace`) defined in the benchmarking con
 e.g. `benchmarkingjob.yaml`). In this quick start, we have done all configurations for you and the interested readers
 can refer to [benchmarkingJob.yaml](https://ianvs.readthedocs.io/en/latest/guides/how-to-test-algorithms.html#step-1-test-environment-preparation) for more details.
 
-The final output might look like this:   
+The final output might look like this:
 
-
-| rank |        algorithm        |      accuracy      |         BWT         |         FWT         |     paradigm     | basemodel |    task_definition     |    task_allocation     | basemodel-learning_rate | basemodel-epochs | task_definition-origins | task_allocation-origins |        time         |                             url                              |
-| :--: | :---------------------: | :----------------: | :-----------------: | :-----------------: | :--------------: | :-------: | :--------------------: | :--------------------: | :---------------------: | :--------------: | :---------------------: | :---------------------: | :-----------------: | :----------------------------------------------------------: |
-|  1   | rfnet_lifelong_learning | 0.2970033189775575 | 0.04239649121511442 | 0.02299711942108413 | lifelonglearning | BaseModel | TaskDefinitionByOrigin | TaskAllocationByOrigin |         0.0001          |        1         |   ['front', 'garden']   |   ['front', 'garden']   | 2023-05-24 15:07:57 | /ianvs/lifelong_learning_bench/robot-workspace-bwt/benchmarkingjob/rfnet_lifelong_learning/efdc47a2-f9fb-11ed-8f8b-0242ac110007 |
+| rank |        algorithm        |      accuracy      |         BWT         |         FWT         |     paradigm     | basemodel |    task_definition    |    task_allocation    | basemodel-learning_rate | basemodel-epochs | task_definition-origins | task_allocation-origins |        time        |                                                               url                                                               |
+| :--: | :---------------------: | :----------------: | :-----------------: | :-----------------: | :--------------: | :-------: | :--------------------: | :--------------------: | :---------------------: | :--------------: | :---------------------: | :---------------------: | :-----------------: | :-----------------------------------------------------------------------------------------------------------------------------: |
+|  1  | rfnet_lifelong_learning | 0.2970033189775575 | 0.04239649121511442 | 0.02299711942108413 | lifelonglearning | BaseModel | TaskDefinitionByOrigin | TaskAllocationByOrigin |         0.0001         |        1        |   ['front', 'garden']   |   ['front', 'garden']   | 2023-05-24 15:07:57 | /ianvs/lifelong_learning_bench/robot-workspace-bwt/benchmarkingjob/rfnet_lifelong_learning/efdc47a2-f9fb-11ed-8f8b-0242ac110007 |
 
 To run the large vision model based cloud-edge collaboration process:
 
-``` shell
+```shell
 cd /ianvs/project/ianvs
 ianvs -f examples/robot/lifelong_learning_bench/benchmarkingjob-sam.yaml
 ```
@@ -153,17 +154,16 @@ e.g. `/ianvs/lifelong_learning_bench/workspace`) defined in the benchmarking con
 e.g. `benchmarkingjob.yaml`). In this quick start, we have done all configurations for you and the interested readers
 can refer to [benchmarkingJob.yaml](https://ianvs.readthedocs.io/en/latest/guides/how-to-test-algorithms.html#step-1-test-environment-preparation) for more details.
 
-The final output might look like this:   
-
+The final output might look like this:
 
-| rank |        algorithm        |      accuracy      |         Task_Avg_Acc         |     paradigm     | basemodel |    task_definition     |    task_allocation     | unseen_sample_recognition | basemodel-learning_rate | basemodel-epochs | task_definition-origins | task_allocation-origins | unseen_sample_recognition-threhold |         time        |                                                               url                                                               |
-|:------:|:-------------------------:|:--------------------:|:---------------------:|:------------------:|:-----------:|:------------------------:|:------------------------:|:-------------------------:|:------------------:|:-------------------------:|:-------------------------:|:---------------------:|:---------------------------------------------------------------------------------------------------------------------------------:|-----------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------:|
-|  1   | sam_rfnet_lifelong_learning | 0.7052917006987501 | 0.6258875117354328 | lifelonglearning | BaseModel | TaskDefinitionByOrigin | TaskAllocationByOrigin | HardSampleMining |          0.0001         |        1         |   ['front', 'garden']   |   ['front', 'garden']   |   0.95   | 2023-08-24 12:43:19 | /ianvs/sam_bench/robot-workspace/benchmarkingjob/sam_rfnet_lifelong_learning/9465c47a-4235-11ee-8519-ec2a724ccd3e |
+| rank |          algorithm          |      accuracy      |    Task_Avg_Acc    |     paradigm     | basemodel |    task_definition    |    task_allocation    | unseen_sample_recognition | basemodel-learning_rate | basemodel-epochs | task_definition-origins | task_allocation-origins | unseen_sample_recognition-threhold | time                |                                                        url                                                        |
+| :--: | :-------------------------: | :----------------: | :----------------: | :--------------: | :-------: | :--------------------: | :--------------------: | :-----------------------: | :---------------------: | :--------------: | :---------------------: | :---------------------: | :--------------------------------: | ------------------- | :---------------------------------------------------------------------------------------------------------------: |
+|  1  | sam_rfnet_lifelong_learning | 0.7052917006987501 | 0.6258875117354328 | lifelonglearning | BaseModel | TaskDefinitionByOrigin | TaskAllocationByOrigin |     HardSampleMining     |         0.0001         |        1        |   ['front', 'garden']   |   ['front', 'garden']   |                0.95                | 2023-08-24 12:43:19 | /ianvs/sam_bench/robot-workspace/benchmarkingjob/sam_rfnet_lifelong_learning/9465c47a-4235-11ee-8519-ec2a724ccd3e |
 
 This ends the quick start experiment.
 
 # What is next
 
-If any problems happen, the user can refer to [the issue page on Github](https://github.com/kubeedge/ianvs/issues) for help and are also welcome to raise any new issue. 
+If any problems happen, the user can refer to [the issue page on Github](https://github.com/kubeedge/ianvs/issues) for help and are also welcome to raise any new issue.
 
-Enjoy your journey on Ianvs!
+Enjoy your journey on Ianvs!

examples/robot/lifelong_learning_bench/testalgorithms/rfnet/RFNet/utils/args.py

@@ -21,7 +21,7 @@ def __init__(self, **kwargs):
         self.gpu_ids = 0
 
         self.seed = 1
-        self.resume = '/home/hsj/ianvs/project/pretrain/pretrain_model.pth'
+        self.resume = None
         self.checkname = 'RFNet'
         self.ft = True
         self.eval_interval = kwargs.get("eval_interval", 50)

examples/robot/lifelong_learning_bench/testalgorithms/rfnet/basemodel-sam.py

@@ -40,16 +40,6 @@ def get_weights(self):
 
     def set_weights(self, weights):
         self.trainer.set_weight(weights)
-
-        epoch_num = 0
-        print("Total epoch: ", epoch_num)
-        loss_all = []
-        for epoch in range(epoch_num):
-            train_loss = self.trainer.my_training(epoch)
-            loss_all.append(train_loss)
-        with open('/home/shijing.hu/ianvs/project/ianvs/train_loss_2.txt', 'a+') as file:
-            np.savetxt(file, loss_all)
-        file.close
 
     def train(self, train_data, valid_data=None, **kwargs):
         self.trainer = Trainer(self.train_args, train_data=train_data)
@@ -86,22 +76,16 @@ def predict(self, data, **kwargs):
         Use the RFNet model to predict at the edge 
         """
         if len(data) > 10:
-            print("predict start for big data")
             my_kwargs = {'num_workers': self.val_args.workers, 'pin_memory': True}
             _, _, self.validator.test_loader, _ = make_data_loader(self.val_args, test_data=data, **my_kwargs)
         else:
-            print("predict start for small data")
             if not isinstance(data[0][0], dict):
                 data = self._preprocess(data)
-                #print("predict starting 69")
             if type(data) is np.ndarray:
                 data = data.tolist()
-                #print("predict starting 72")
-            #print("predict starting 73")
             self.validator.test_loader = DataLoader(data, batch_size=self.val_args.test_batch_size, shuffle=False,
                                                 pin_memory=True)
 
-        #print("predict starting 75")
         predictions, scores = self.validator.validate()
         return predictions
 
@@ -110,22 +94,16 @@ def predict_cloud(self, data, **kwargs):
         Use the SAM model to predict at the cloud 
         """
         if len(data) > 10:
-            print("predict start for big data")
             my_kwargs = {'num_workers': self.val_args.workers, 'pin_memory': True}
             _, _, self.validator.test_loader, _ = make_data_loader(self.val_args, test_data=data, **my_kwargs)
         else:
-            print("predict start for small data")
             if not isinstance(data[0][0], dict):
                 data = self._preprocess(data)
-                #print("predict starting 69")
             if type(data) is np.ndarray:
                 data = data.tolist()
-                #print("predict starting 72")
-            #print("predict starting 73")
             self.validator.test_loader = DataLoader(data, batch_size=self.val_args.test_batch_size, shuffle=False,
                                                 pin_memory=True)
 
-        #print("predict starting 75")
         predictions = self.validator.validate_cloud()
         return predictions
 
@@ -134,27 +112,20 @@ def predict_score(self, data, **kwargs):
         Get the prediction scores of RFNet model 
         """
         if len(data) > 10:
-            print("predict start for big data")
             my_kwargs = {'num_workers': self.val_args.workers, 'pin_memory': True}
             _, _, self.validator.test_loader, _ = make_data_loader(self.val_args, test_data=data, **my_kwargs)
         else:
-            print("predict start for small data")
             if not isinstance(data[0][0], dict):
                 data = self._preprocess(data)
-                #print("predict starting 69")
             if type(data) is np.ndarray:
                 data = data.tolist()
-                #print("predict starting 72")
-            #print("predict starting 73")
             self.validator.test_loader = DataLoader(data, batch_size=self.val_args.test_batch_size, shuffle=False,
                                                 pin_memory=True)
 
-        #print("predict starting 75")
         predictions, scores = self.validator.validate()
         return scores
 
     def evaluate(self, data, **kwargs):
-        #print("evaluate starting 77")
         self.val_args.save_predicted_image = kwargs.get("save_predicted_image", True)
         samples = self._preprocess(data.x)
         predictions = self.predict(samples)