Skip to content

Sm3ni is an Optical Musical Recognition project written in python that converts music sheets images to a text file representing the musical notes then to a .wav audio file that represents the music sheet .

License

Notifications You must be signed in to change notification settings

MENNAELMESALMY/Sm3ni

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Sm3ni Logo

GitHub forks GitHub Repo stars GitHub contributors GitHub issues

Table of Contents

  1. Description
  2. Pipeline
  3. Tools
  4. Run
  5. Contributing
  6. Contributors
  7. License

Description

Sm3ni is an Optical Musical Recognition project written in python that converts music sheets images to a text file representing the musical notes then to a .wav audio file that represents the music sheet .

This repo will contain our used pipeline for the OMR based on different cases for the input images .

Pipeline

  • Preprocessing

    Preprocessing

    • Smoothing

      We first apply a smoothing test by calculating the signal to noise ratio in the image .

      If the SNR is above some threshold the bilateral smoothing filter is applied on the image

    • Illumination

      Uneven illumination test is applied on the image

      If the image is unevenly illuminated a contrast enhancement algorithm called screened poisson contrast enhancement is applied on the image

    • Binarization

      If the image is unevenly illuminated we apply Feng Local Thresholding for low quality documented images.

      If the image is evenly illuminated we apply OTSU Local Thresholding .

    • Deskewing

      Rotate the image from 0 to 360 degree then find the angle of maximum horizontal projection sum and rotate the image with the right angle .

      Preprocessing Outputs

  • Staff-Lines Removal

    Staff-Removal

    Horizontal/Non-horizontal test is first applied on the image then 2 different techniques are applied based on the test.

    • Horizontal Staff-Lines Removal

      Get horizontal projection of the image

      Find the peaks and get the staff-height and staff-space

      Remove lines with width same as the peaks' width

    • Non-Horizontal Staff-Lines Removal

      Apply run length encoding .

      Find the mod of consecutive black pixels to be the staff height and mod of consecutive white pixels to be the staff-space.

      Remove all regions with height more than the staff height.

      Get the removed lines image by subtracting the removed symbols image from the original image.

    Staff-Removal Output

  • Segmentation

    Segmentation

    Segmenting the image into staff-segments.

    Segmenting each staff segment to the musical symbols.

    Detect Clef position and rotate the whole image 180 degree if it exists in the bottom right corner.

    For non horizontal images de-skew each symbol from -45 to 45 degree till getting the right angle for rotation from projection sum .

  • Features Extraction

    From this step we get a feature vector describing each symbol that will be used later in the classification .

    • Stems

      Detect stems in the symbol by vertical projection then remove them.

    • Notes-Heads

      Get all regions left in the symbol and threshold on each region by solidity,eccentricity,area,width and height with respect to the staff-space

      count all valid heads to be number of black heads .

      Apply region filling to the symbol then repeat the previous steps.

      count all valid heads to be number of white heads.

    • Beams

      Get all regions left in the symbol then apply thresholding by aspect ratio to detect number of beams lines.

    • UporDown

      Detect whether notes is up or down by comparing the center of the highest head with the tallest stem in the symbol .

    • Flags

      Apply skeletonization to the symbol.

      Find all connected right down/left up paths with thresholding on the path's length.

      Count number of valid flags.

  • Classification

    Classification

  • Generating Output

    Output

    • Sample Text Output

      Text

    • Sample Audio Output

      Audio.wav

Tools

  • Python
  • Numpy
  • Skimage
  • Opencv
  • Os : Linux-Ubuntu

Run

$ conda env create -f requirements.yml
$ conda activate omrproject
$ python src/main.py <Input Folder Absolute Path> <Output Folder Absolute Path>

Contributing

1. Fork this repo
2. Create new branch
	$ git checkout -b <YourBranch>
3. Add your modifications then
	$ git commit -m "Commit Message"
	$ git push origin <YourBranch>
4. Create PR

Stay in touch

License

Sm3ni is MIT licensed.

About

Sm3ni is an Optical Musical Recognition project written in python that converts music sheets images to a text file representing the musical notes then to a .wav audio file that represents the music sheet .

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%