Skip to content

Benchmarks for the OMEinsum.jl package and einsum in python

Notifications You must be signed in to change notification settings

under-Peter/OMEinsum-Benchmarks

Repository files navigation

OMEinsum-Benchmarks

This repository is used to compare the performance of einsum in python (numpy, torch), with implementations in julia.

In /pythonscripts, the benchmarking-scripts for numpy.einsum and torch.einsum can be found. The tested operations and the sizes of the arrays involved are:

label description einsum-equation small medium large
matmul matrix multiplication ij,jk -> ik 10×10 100×100 1000×1000
dot dot product ijl,ijl-> 10×10×10 20×20×20 50×50×50
ptrace partial trace iij->j 10×10×10 20×20×20 50×50×50
hadamard the hadamard product ijk,ijk->ijk 10×10×10 30×30×30 50×50×50
tcontract tensor contraction ikl,kjl->ij 2×2×2 10×10×10 30×30×30
indexsum summing over a (single) index ijk->ik 10×10×10 30×30×30 100×100×100
trace matrix-trace ii-> 10×10 100×100 1000×1000
diag taking the diagonal w.r.t to two or more indices jii->ji 10×10×10 20×20×20 50×50×50
starandcontract mixing tensor and star-contraction ij,il,il->j 10×10 30×30 100×100
star starcontracting over a single index ij,ik,il->jkl 10×10 30×30 100×100
outer taking the outer product between two tensors ij,kl->ijkl 10×10 50×50 100×100
batchmul batchmultiplying two batches of matrices ijl,jkl->ikl 10×10×3 100×100×3 1000×1000×3
perm permuting a tensors indices ijkl->ljki 2×2×2×2 10×10×10×10 30×30×30×30

The benchmarks have been evaluated over inputs of different size, labelled (ad-hoc) as tiny, small, medium, large and huge. The current benchmark has been run with torch, numpy for python and OMEinsum.jl and Einsum.jl for julia.

Overview

The plots are split by the memory footprint of the operations, i.e. by size size of the tensors involved which have been classified as tiny, small, medium, large or huge rather arbitrarily. The results below are for tensors of type float64 but more results can be found in the plots/ folder. Generally, changing to float32 or complex types does not change the picture too much, except that torch does not support complex numbers.

small tensors

medium tensors

large tensors

About

Benchmarks for the OMEinsum.jl package and einsum in python

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published