Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add readme about dataset labeling #220

Merged
merged 8 commits into from
Jul 3, 2018

Conversation

sagarvijaygupta
Copy link
Collaborator

No description provided.

@codecov-io
Copy link

codecov-io commented Jun 21, 2018

Codecov Report

Merging #220 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##           master    #220   +/-   ##
======================================
  Coverage    20.8%   20.8%           
======================================
  Files          11      11           
  Lines        1216    1216           
  Branches      160     160           
======================================
  Hits          253     253           
  Misses        961     961           
  Partials        2       2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b18eae0...090babf. Read the comment docs.


---

#### They look same.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They look the same

@@ -0,0 +1,56 @@

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should move everything about labeling in this document, put it in the top-level directory calling it "LABELING.MD", and referencing it in README.md.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, that means we need to completely remove the labeling part in README.md, and put a reference over there to "LABELING.md" which will be in the top-level directory, keeping images in labeling_guide folder only (without the ReadMe.md file which is there presently).

# Labeling Guidelines

## Images marked as compatible - y

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove the newlines around the --- lines.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(if possible)

---

#### They are different.
#### Mark the parts which are different logically with Green.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't mention the color here, we are going to mention it in the section where we explain how the labeling should be performed.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which are logically different

---

#### They look different.
#### Mark the parts which are same logically with Yellow.
Copy link
Owner

@marco-c marco-c Jun 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which are logically the same


#### They look different.
#### Mark the parts which are same logically with Yellow.
>Different advertisements, different videos loaded, time-in-clock are marked
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's time-in-clock?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A screenshot which has present time displayed, it will be different in two screenshots as they are taken at different times but still compatible.

@marco-c marco-c changed the title Readme added for labeling of dataset. Add readme about dataset labeling Jun 24, 2018
LABELING.md Outdated
#### They look different.
#### Mark the parts which are logically the same.
>Different advertisements, different videos loaded, time-in-clock, captchas are marked
as different yet compatible.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also add the example which was in the readme about the news site.

Different videos loaded shouldn't be there, the same video should be loaded in both browsers if the site is compatible. You can say different frames of a video loaded.
Rename time-in-clock as it is unclear.

I would say something like Since the screenshots are taken at different times in the two browsers, there are differences which are not incompatibilities but are actually due to the different timing. For example, a banner could be showing a different advertisement, a video could be in two different frames, a clock could be showing different time, a captcha could be showing different characters or images, two news could be different, and so on.

LABELING.md Outdated
#### Couples of images which are not compatible
#### They are different.
#### Mark the parts which are logically different.
> Improper loading of images, missing text, different design, different languages are marked incompatible.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, improper loading ...

Add for example because these are not the only cases.

marco-c
marco-c previously approved these changes Jul 3, 2018
@marco-c
Copy link
Owner

marco-c commented Jul 3, 2018

Thanks! I missed you made the updates to the PR... There are some conflicts though.

@marco-c
Copy link
Owner

marco-c commented Jul 3, 2018

The colors are wrong now, as we changed them in #221.

@sagarvijaygupta
Copy link
Collaborator Author

@marco-c I will have to update the images in that case!

@sagarvijaygupta
Copy link
Collaborator Author

@marco-c I have changed the images accordingly.

@marco-c marco-c merged commit e43bdb0 into marco-c:master Jul 3, 2018
@propr
Copy link

propr bot commented Jul 3, 2018

Please provide your feedback on this pull request here.

Privacy statement: We don't store any personal information such as your email address or name. We ask for GitHub authentication as an anonymous identifier to account for duplicate feedback entries and to see people specific preferences.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants