Skip to content

http server supporting chunked uploads and downloads using transfer-encoding chunked

License

Notifications You must be signed in to change notification settings

mjneil/go-chunked-streaming-server

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

36 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

go-chunked-streaming-server

This is simple webserver written in GoLang that allows you to chunk transfer from the ingress and egress sides. In other words:

  • If it receives a GET request for object A, and object A is still being ingested, we send all we have of object A and we keep that GET request open sending on real time (and in memory) all the chunks that we received for object A, until that object is closed.
  • If we receives a GET request for object B and this object is already written (and closed), we send the object B as any other static web server.

Ideal for low latency media processing pipelines.

Usage

Installation

  1. Just download GO in your computer. See GoLang
  2. Create a Go directory to be used as a workspace for all go code, i.e.
mkdir ~/MYDIR/go
  1. Add GOPATH to your ~/.bash_profile or equivalent for your shell
export GOPATH="$HOME/MYDIR/go"
  1. Add GOPATH/bin to your path in ~/.bash_profile or equivalent for your shell
export PATH="$PATH:$GOPATH/bin
  1. Restart your terminal or source your profile
  2. Clone this repo:
go get github.com/mjneil/go-chunked-streaming-server
  1. Go the the source code dir `
cd $HOME/MYDIR/go/src/github.com/mjneil/go-chunked-streaming-server
  1. Compile main.go doing:
make

Testing

You can execute ./bin/./go-chunked-streaming-server -h to see all the possible command arguments.

Usage of ./bin/go-chunked-streaming-server:
  -c string
        Certificate file path (only for https)
  -d    Indicates to remove files from the server based on original Cache-Control (max-age) header
  -i int
        Port used for HTTP ingress/ egress (default 9094)
  -k string
        Key file path (only for https)
  -o string
        JSON file path with the CORS headers definition
  -p string
        Path used to store (default "./content")
  -r    Indicates DO NOT use disc as persistent/fallback storage (only RAM)

Example simple HTTP

  • Start the server
./bin/./go-chunked-streaming-server
  • Upload a file
curl http://localhost:9094 --upload-file file.test.txt
  • Consume the file and saved it to disc
curl http://localhost:9094/file.test.txt -o file.test.downloaded.txt

Example chunked HTTP

We could build a low latency HLS pipeline in conjunction with go-ts-segmenter by doing:

  1. Start webserver
./bin/./go-chunked-streaming-server
  1. Generate LHLS data
ffmpeg -f lavfi -re -i smptebars=size=320x200:rate=30 -f lavfi -i sine=frequency=1000:sample_rate=48000 -pix_fmt yuv420p -c:v libx264 -b:v 180k -g 60 -keyint_min 60 -profile:v baseline -preset veryfast -c:a aac -b:a 96k -f mpegts - | manifest-generator -l 3 -d 2
  1. Play that data with any HLS player (recommended any player that implements low latency mode)
ffplay http://localhost:9094/results/chunklist.m3u8

Or use Safari with this URL http://localhost:9094/results/chunklist.m3u8

About

http server supporting chunked uploads and downloads using transfer-encoding chunked

Resources

License

Stars

Watchers

Forks

Packages

No packages published