towhee-0.9.0

Towhee.command

• Support operators code package and upload to pypi
  • towhee package -n [namespace] -p [repo_name]
• Support weight files upload to S3, list files in S3 folder and download from S3
  • towhee lsS3 -p [path in bucket]
  • towhee downloadS3 -pl [path local] -pb [path bucket]

s3: model weights

• 5 model weights on S3: clip4clip, vggish, nnfp_fma, nnfp_fma_distill_128

New Models

• 1 Video classification model
  • Vis4mer
• 3 SOTA vision backbones
  • MCProp
  • RepLKNet
  • Shunted Transformer

Operators

• Add 2 code search operators: code_search.codebert, code_search.unixcoder
• Add 5 image captioning operators: image_captioning.expansionnet-v2, image_captioning.magic, image_captioning.clip_caption_reward, image_captioning.blip, image_captioning.clipcap
• Add 5 image-text embedding operators: image_text_embedding.albef, image_text_embedding.ru_clip, image_text_embedding.japanese_clip, image_text_embedding.taiyi, image_text_embedding.slip
• Add 1 machine-translation operator: machine_translation.opus_mt
• Add 1 filter-tiny-segments operator: video-copy-detection.filter-tiny-segments

Notebooks

• Update the beginner tutorial of audio fingerprinting: Audio Fingerprint I: Build a Demo with Towhee & Milvus (modified vote method)
• Add an advanced tutorial for audio fingerprinting: Audio Fingerprint II: Music Detection with Temporal Localization (increased accuracy from 84% to 90%)

Performance

• Improve video de-copy F1 metrics on MPAA dataset.(88.83% -> 94.46%)

Towhee

• New pipeline interface. The pipeline nodes support map/flat_map/filter/window/time_window/window_all types.