Store annotations in database and retrieve them later

[salif-04]

I want to draw red rectangular annotations on the images and store them in the database. Later when that image is viewed again, those annotations should be retrieved and viewed. Is there a way by which I can target these annotations, and further redraw them.

[ivmartel]

Yes, the application can save a state that contains annotations, see onStateSave in application.js#L1021. You can also have a look at the tests to further understand.

[salif-04]

From the examples, I got to know that those annotations will be saved locally as a json file. Is it so?

What I want is to use this in my application, which views mammograms(dicoms), add annotations on them and store the annotations in the database rather than generating the json file.

Could you please help me with the setup?

I would be using dwv-jqui. Should I raise an issue over there or we can discuss over here?

[ivmartel]

We can continue discussing here.

To create the state you could use this code:

var state = new dwv.State();
var json = state.toJSON(app);

You would then have to write the code that sends the state to your server.

[salif-04]

I made a simple page, included the dicom and added zoom and pan functionality.
https://github.com/salif-04/tmp/blob/master/index.php

Now how to add the toolbox.

Error:
** Cannot read property 'getDrawLayer' of null at State.toJSON**

[ivmartel]

Check out the code of dwv-jqui, copy all and remove what you do not need.

[salif-04]

Loaded the image from url #489.
Then I removed the history window. After removing the history window, it showed errors.
Uncaught TypeError: Cannot read property 'hasChildNodes' of null

[ivmartel]

Can you share the full console log to have more details on the error?

[salif-04]

Sure.

[ivmartel]

You also need to remove the undo gui from the app configuration, see applauncher.js#L16.

[salif-04]

Still the same error.

[salif-04]

[anoob09]

@ivmartel
About saving the state.json file locally.
Where do I need to make the changes that you mentioned above ?
Should I change dwv.min.js file or I can create a new JavaScript function ?

[salif-04]

Ok. Now I am done with loading the image.
Just one more help.

I want to rename the json file before downloading it. Means I dont want that dialog box thats prompts us for the download. I want to provide a specific name and download the json file in background.

[ivmartel]

@anoob09 The code I mention above can go in an event handler that you create, so in another js file than the main dwv one.
@salif-04 How did you fix your error? If you send data to a server, I guess you will be able to store it as you want. The name of the file comes from the html link, see appgui.js#L196 for the dwv-jqui.

[anoob09]

@ivmartel I used the simplest dwv from your js-fiddle and then passed the URL and added the jpg and jpeg decoders for good measure.

[anoob09]

I have added a new button with download_json . I have added onclick event listener. This is the code for the button
var state = new dwv.State(); var new_json = state.toJSON(app); var myJSON = JSON.stringify(new_json); jQuery.post("add_annotaion_script.php", {myJSON:myJSON} );
I am getting the following error after clicking on it.
Uncaught ReferenceError: app is not defined at HTMLButtonElement.document.getElementById.onclick
After going through the event listener of the default Download button. I found the following code. Can you break it down for me please ?
this.onStateSave=function(){var e=new H.State;o.getElement("download-state").href="data:application/json,"+e.toJSON(o)}

[salif-04]

@ivmartel I dynamically opened the image #489 and it worked.

Now since I loaded the image dynamically, what object/parameter do we need to pass to state.toJSON() in the eventListener of a button defined to send data to server.

var state = new dwv.State();
var json = state.toJSON(app);

[anoob09]

Tried changing the code for dwv.min.js since that's where this code is present
this.onStateSave=function(){var e=new H.State;o.getElement("download-state").href="data:application/json,"+e.toJSON(o)}
But for some reason all the changes I make are reverted. When I open dwv.min.js via editor, it shows that code has been changed. But when I inspect code for the Download Button after launching app, I don't see the changes I made. It shows the default dwv.min.js code.

[salif-04]

We loaded the dicom, made the state.file but when we are trying to load them, it shows an error:
Error: Unknown state file format version: 'undefined'.
at State.fromJSON (dwv.min.js:11)
at H.io.JSONTextLoader.R.t.onload (dwv.min.js:2)
at H.io.JSONTextLoader.load (dwv.min.js:11)
at XMLHttpRequest. (dwv.min.js:11)

What can be the reason Sir?

[ivmartel]

The application onStateSave is used if you bind it to an event. It updated the download link with the content of the state (o.getElement("download-state").href=...). You are quoting the code of the minified version of dwv, replace it with the non minified version to better understand the code.

@anoob09, are you running one of the watch scripts provided with dwv? If so, it is possible that they re-create the minified dwv at each code change.
In the code you share there is no need to call JSON.stringify(new_json). I think this is why @salif-04 is getting the Unknown state file format error.

Are you two working together? And no need for the sir, Sir!!

[salif-04]

Yes, we are working togeather.

[anoob09]

@ivmartel We tried it without using JSON.stringify(new_json) but it didn't work, intsead, it put double qoute (") in the JSON file which is invalid. If we use stringify then we are able to create the same JSON file as we get from the download link. So I don't think the error is due to JSON file. Its something else.

[salif-04]

What we are doing is
dwv.min.js:

this.onStateSave=function(){
var e=new H.State;
var new_json = e.toJSON(o);
jQuery.post("add_annotation_script.php",
            {myJSON:new_json}
            );
}

add_annotation_script.php:

$myJSON = json_decode($_POST['myJSON']);
$rad_id = $_SESSION['rad_id'];
$image = $_SESSION['image'];
$case_id = $_SESSION['case_id'];
$view_id = $_SESSION['view_id'];
if (!file_exists("anno/".$rad_id)) {
    mkdir("anno/".$rad_id,0777,false);    
}
$path = "C:/wamp64/www/MIC/anno/".$rad_id."/".$image.$case_id.".json";
$f = fopen($path, "w+");
fwrite($f, $myJSON);
fclose($f);
chmod($path, 0777);

[salif-04]

We are getting the annotations back in the history window(not all, some of the annotations are missing) but these annotations are not visible by default on the dicom. They are visible after chicking on them from the showAnnotation tab.

Alert :
TypeError: Cannot read property 'getParent' of undefined

Error :
TypeError: Cannot read property 'getParent' of undefined (dwv.min.js:2)
at Konva.Group.add (konva.min.js:43)
at H.DrawController.setDrawings (dwv.min.js:11)
at App.setDrawings (dwv.min.js:2)
at State.apply (dwv.min.js:11)
at H.io.JSONTextLoader.R.t.onload (dwv.min.js:2)
at H.io.JSONTextLoader.load (dwv.min.js:11)
at XMLHttpRequest. (dwv.min.js:11)

We are loading the state.json file through URL
http://localhost/MIC/annotation.php?input=http://localhost/MIC/mammos/case0006/000137.dcm&state=http://localhost/MIC/anno/5/LMLO6.json&case_id=6&image=LMLO&type=report&view_id=5

Firefox says:
Alert: TypeError: t is undefined

[anoob09]

`dwv.min.js:2 TypeError: Cannot read property 'getParent' of undefined

at Konva.Group.add (konva.min.js:43)

at H.DrawController.setDrawings (dwv.min.js:8)

at App.setDrawings (dwv.min.js:2)

at State.apply (dwv.min.js:8)

at H.io.JSONTextLoader.R.t.onload (dwv.min.js:2)

at H.io.JSONTextLoader.load (dwv.min.js:8)

at XMLHttpRequest.<anonymous> (dwv.min.js:8)

G @ dwv.min.js:2Getting this error while loading the JSON file via URL. I tried using thedwv.jsfile instead ofdwv.min.js. It worked at first. But when I reloaded the page I got the same error. After that, when switched back to dwv.min.jsand it worked again. But when I reloaded the page, I got the same error. I don't thinkJSONfile has anything to do here. Are you storing theJSON` files in the browser via cookies or something ?

[ivmartel]

It seems Konva has troubles loading the json as if it does not contain the information it expects. Have you tried to load the state manually to see if the file is ok? No cookies, just a json file.

[anoob09]

How to load it manually? And I have verified that the json file created via download link and json file created by me are same.

[ivmartel]

You can drag and drop it on the dwv-jqmobile demo for example (after having loaded the DICOM).

[anoob09]

{ "version": "0.3", "window-center": 2047, "window-width": 4096, "position": { "i": 0, "j": 0, "k": 0 }, "scale": 1, "scaleCenter": { "x": 0, "y": 0 }, "translation": { "x": 0, "y": 0 }, "drawings": { "attrs": { "listening": true, "visible": true }, "className": "Layer", "children": [{ "attrs": { "name": "position-group", "id": "slice-0_frame-0", "visible": true }, "className": "Group", "children": [{ "attrs": { "name": "line-group", "visible": true, "id": "e17db3o4bhkr8", "draggable": true }, "className": "Group", "children": [{ "attrs": { "points": [38, 499, 2530, 499], "stroke": "#ffff80", "strokeWidth": 12.974658869395713, "name": "shape" }, "className": "Line" }, { "attrs": { "x": 43, "y": 499, "sides": 3, "radius": 5, "rotation": -90, "fill": "#ffff80", "strokeWidth": 12.974658869395713, "name": "shape-triangle" }, "className": "RegularPolygon" }, { "attrs": { "x": 2505, "y": 506.5, "name": "label" }, "className": "Label", "children": [{ "attrs": { "fontSize": 77.84795321637428, "fontFamily": "Verdana", "fill": "#ffff80", "name": "text" }, "className": "Text" }, { "attrs": { "height": 77.84795321637428 }, "className": "Tag" }] }] }, { "attrs": { "name": "line-group", "visible": true, "id": "xlqbq0roptj1k", "draggable": true }, "className": "Group", "children": [{ "attrs": { "points": [51, 1349, 2549, 1316], "stroke": "#ffff80", "strokeWidth": 12.974658869395713, "name": "shape" }, "className": "Line" }, { "attrs": { "x": 55.99956375930118, "y": 1348.9339529207139, "sides": 3, "radius": 5, "rotation": -90.75686579029184, "fill": "#ffff80", "strokeWidth": 12.974658869395713, "name": "shape-triangle" }, "className": "RegularPolygon" }, { "attrs": { "x": 2524, "y": 1301, "name": "label" }, "className": "Label", "children": [{ "attrs": { "fontSize": 77.84795321637428, "fontFamily": "Verdana", "fill": "#ffff80", "name": "text" }, "className": "Text" }, { "attrs": { "height": 77.84795321637428 }, "className": "Tag" }] }] }, { "attrs": { "name": "line-group", "visible": true, "id": "jy68jas9zz6vi", "draggable": true }, "className": "Group", "children": [{ "attrs": { "points": [58, 2426, 2179, 2387], "stroke": "#ffff80", "strokeWidth": 12.974658869395713, "name": "shape" }, "className": "Line" }, { "attrs": { "x": 62.99915495900947, "y": 2425.908077773031, "sides": 3, "radius": 5, "rotation": -91.05341047542493, "fill": "#ffff80", "strokeWidth": 12.974658869395713, "name": "shape-triangle" }, "className": "RegularPolygon" }, { "attrs": { "x": 2154, "y": 2372, "name": "label" }, "className": "Label", "children": [{ "attrs": { "fontSize": 77.84795321637428, "fontFamily": "Verdana", "fill": "#ffff80", "name": "text" }, "className": "Text" }, { "attrs": { "height": 77.84795321637428 }, "className": "Tag" }] }] }] }] }, "drawingsDetails": { "e17db3o4bhkr8": { "textExpr": "", "longText": "", "quant": null }, "xlqbq0roptj1k": { "textExpr": "", "longText": "", "quant": null }, "jy68jas9zz6vi": { "textExpr": "", "longText": "", "quant": null } } }

I drew 3 lines in this dicom. But when I used the state.json file. The 1st and 3rd drawings were retrieved. That too, the lines weren't visible at first. I had to select them from history window. But after that I wasn't able to edit these lines. To edit these I had to add one more line, and then I was able to edit these lines. It's very buggy. You can see the state.json file.

[salif-04]

No. We are not using yarn or npm scripts.

We simply linked all the js files to our php page.

[salif-04]

I took the live demo of dwv-jqmobile and dwv-jqui. Loaded the image, made some annotations on them, downloaded the image. Then again I loaded the image and tried to load the state(those I downloaded) manually, it showed the same error in both(dwv-jqmobile and dwv-jqui).

[ivmartel]

There seems to be something dwv does not like about your images... I'll investigate and tell you.

[ivmartel]

Ok, good one, it's not only your images, it's a general bug! State files with multiple shapes crash the app.

Can you try replacing this line drawController.js#L395 with var stateGroup = statePosKids[0]; (changing the array index from j to 0). It seems that shapes cannot belong to multiple groups, so if you affect a shape to a group it gets removed from the one it belong to before.

Tell me how it goes and thanks for spotting it!

[salif-04]

We cannot locate the drawController.js in dwv-jqui.

[ivmartel]

It's in dwv.

[anoob09]

It worked!
But now while declaring this in dwv
var state = new dwv.State(); var json = state.toJSON(app);
I am getting an error saying app is not defined. How to resolve this?

[ivmartel]

You need to call this code in the same context as where you create the app, in the demos that would be in the applauncher files.

[salif-04]

It worked well. Thanks for all the help. We appreciated it.

[salif-04]

The application cannot cannot distinguish between a arrow and a line.
When we are drawing an arrow and saving the state; on reloading the state arrows are showed as lines.
Rest all shapes are working well.

[salif-04]

We are also unable to delete any of the annotations.

We are trying to delete by dragging and dropping the annotations to the red cross mark but it is not working.
Not working even in the live demo of jqui.

[anoob09]

You need to check edit mode after loading the annotations through JSON to delete it.

[salif-04]

No. There we have Delete All option. I'm taking about deleting the annotations individually.

[ivmartel]

For the arrow, I found a fix: change this line in arrow.js#L61 to
var radius = 5 * style.getScaledStrokeWidth();
(change the number to fit your preferred size).

I'll have a look at the delete problem...

[salif-04]

Yes. Now the arrow head is visible.

[salif-04]

Did you find any fix to the delete problem?

[anoob09]

Is there any way to reduce the image load time? Chrome is taking 3.7-4 seconds to load the image.

[ivmartel]

Nearly there for the delete problem.

Improving the loading time on the dwv side needs work... Your image is quite large, can you make it smaller?

[anoob09]

I don't think we can compromise with the size here. Please let us know what can we to reduce the loading time.

[anoob09]

@ivmartel Is there any way to reduce the loading time ? I just want to show the image from DICOM file. I don't want to add any features.

[ivmartel]

Understood but I do not know of any easy solution... One thing would be to check that there are no bottlenecks in the code and that the code is the fastest it could be. Parsing the pixel data using web workers could help, this needs investigation...

[salif-04]

Hey. Did you fix the delete issue?

[ivmartel]

Yep cf #517

[salif-04]

Now its working well.

[ivmartel]

Good. Can you close the issue if you think it is resolved?