Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix benchmark code and rerun #33

Merged
merged 2 commits into from
Nov 28, 2020
Merged

fix benchmark code and rerun #33

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
50d4cc9
fix benchmark code and rerun
mileslucas Nov 28, 2020
0faf89e
update benchmark info in paper
mileslucas Nov 28, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions bench/adi.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,16 @@ end

vip = pyimport("vip_hci")
py_algs = (
vip.medsub.medsub,
(args...; kwargs...) -> vip.pca.pca(args...; ncomp=20,kwargs...),
(args...; kwargs...) -> vip.nmf.nmf(args...; ncomp=20, kwargs...),
(args...; kwargs...) -> vip.medsub.median_sub(args..., verbose=false, kwargs...),
(args...; kwargs...) -> vip.pca.pca(args...; ncomp=20, verbose=false, kwargs...),
(args...; kwargs...) -> vip.nmf.nmf(args...; ncomp=20, verbose=false, kwargs...),
)

for dataset in (BetaPictoris, HR8799)
@info "Benchmarking - $dataset ADI Reduction"
cube = dataset[:cube]
angles = dataset[:pa]
for (alg, name) in zip(julia_algs, ("median", "pca_20", "nmf_20"))
for (alg, name) in zip(py_algs, ("median", "pca_20", "nmf_20"))
(name == "nmf_20" && dataset === HR8799) && continue
@info "Python - $alg"

Expand Down
20 changes: 10 additions & 10 deletions bench/adi_benchmarks.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
framework,alg,N,time
ADI.jl,median,622261,0.035648202
ADI.jl,median,18574074,1.491676292
ADI.jl,nmf_20,622261,35.85430145
ADI.jl,pca_20,622261,0.050104434
ADI.jl,pca_20,18574074,2.91678003
VIP,median,622261,0.038019478
VIP,median,18574074,1.271641821
VIP,nmf_20,622261,39.894243642
VIP,pca_20,622261,0.054617163
VIP,pca_20,18574074,2.936397107
ADI.jl,median,622261,0.037190855
ADI.jl,pca_20,622261,0.053404161
ADI.jl,nmf_20,622261,41.172108408
ADI.jl,median,18574074,1.305717409
ADI.jl,pca_20,18574074,3.093096529
VIP,median,622261,0.045492888
VIP,pca_20,622261,0.066249575
VIP,nmf_20,622261,0.92181125
VIP,median,18574074,1.094448966
VIP,pca_20,18574074,3.142223225
2 changes: 1 addition & 1 deletion paper/paper.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Currently `ADI.jl` supports full-frame ADI and RDI processing, with experimental

High-contrast imaging as a field predominantly utilizes Python for data reduction. We break down some of the necessary computations into *pre-processing*, which includes raw calibration of data, the centering and stacking of the data cube, bad-pixel removal, etc., *post-processing*, which includes the PSF approximation and subtraction, *posterior analysis* which includes analyzing the results of post-processing like detection statistics and throughput calibration, and finally *forward modeling* which includes various statistical models for companions and disks for use with post-processing algorithms. `ADI.jl` primarily focuses on post-processing and posterior analysis.

Some notable libraries for HCI tasks include the Vortex Imaging Pipeline (`VIP`) [@gomez_gonzalez_vip_2016], `pyKLIP` [@2015ascl.soft06001W], and `PynPoint` [@pynpoint:2019]. A table of the feature sets of these packages alongside `ADI.jl` is presented below. In particular, `VIP` has served as a useful source of information regarding HCI image-processing as well as detailed implementations of common ADI algorithms. This has been indispensable in the development of `ADI.jl`, although this package is not a direct translation. In our small benchmark suite, `ADI.jl` is roughly the same speed or slightly quicker than `VIP` for the algorithms we tested, while detection maps were ~2 orders of magnitude quicker.
Some notable libraries for HCI tasks include the Vortex Imaging Pipeline (`VIP`) [@gomez_gonzalez_vip_2016], `pyKLIP` [@2015ascl.soft06001W], and `PynPoint` [@pynpoint:2019]. A table of the feature sets of these packages alongside `ADI.jl` is presented below. In particular, `VIP` has served as a useful source of information regarding HCI image-processing as well as detailed implementations of common ADI algorithms. This has been indispensable in the development of `ADI.jl`, although this package is not a direct translation. In our small benchmark suite, `ADI.jl` is roughly the same speed or slightly quicker than `VIP` for the algorithms we tested, except for NMF, while detection maps were ~2 orders of magnitude quicker.

In general `VIP` offers the most diversity in algorithms and their applications, but not all algorithms are as feature-complete as the PCA implementation. `VIP` also contains many useful utilities for pre-processing and a pipeline framework. `pyKLIP` only uses the PCA algorithm, but offers many forward modeling implementations. `PynPoint` has a highly modular pre-processing module that is focused on pipelines.

Expand Down