-
Notifications
You must be signed in to change notification settings - Fork 19
/
RELEASE_NOTES
596 lines (469 loc) · 18.8 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
0.6.15 (not yet released)
------
Enhancements
- numpy_util.match works for non-integer data types
0.6.14
------
Enhancements
- Add code changes to C extensions to compile on windows.
Note recfile compiles but failst the unit tests, which
are skipped for windows CI.
- Add windows CI
- remove unused _stat_util.c extension
0.6.13
------
Comapatibility
- More numpy 2 updates
0.6.12
----------
Enhancements
- use stable sort in histogram
Bug Fixes
- Fixed bug in inverting WCS to map ra,dec to x,y near ra = 0
(Matt Becker)
- fix splitarray corner cases
Comapatibility
- update for numpy 2 (Eli Rykoff)
- update to new scipy usage for scipy.integrate.cumulative_trapezoid
v0.6.11
--------------------------
Bug Fixes
- use pyproject.toml (Jo Bovy)
- Fix installation on newer systems (Jo Bovy)
v0.6.10
--------------------------
Bug Fixes
- Added numpy to install_requires in setup.py
- Fixed building on GCC11
v0.6.9
-------------------------
Removed stomp_util module; stomp was abandoned years
ago and the name stomp conflicts with a new package
that does something different
New Features
- add function esutil.numpy_util.combine_fields, which
combines the fields from a set of arrays
- Add get_ntriangles method to HTM objects
Bug Fixes
- Fix bugs for 2.7 syntax
- Fixed bug in lazy WCS distorion inverse computation.
v0.6.8
-------------------------
Bug Fixes
- hdfs py3 fixes
- protect stomp import
- fix color keyword in mwhiskers
v0.6.7
-------------------
New Features
- randsphere, randcap take optional rng
- converted unit over to pytest style. Also moved out various tests from
individual modules and sub packages
- move to setuptools
Bug Fixes
- coords sphdist and gcirc return zero distance of ra, dec match
exactly
- made htm distance always zero when ra, dec exactly match
v0.6.6
--------------------------
Bug Fixes
- Fixed wrong WCS jacobian elements when ra wraps around zero
v0.6.5
--------------------------
New Features
- Added ability to pickle cosmology objects (Jacob Ider Chitham)
- in wcs, change inverse computation to lazy-computation speeding up
initialization.
Compatability
- reran the SWIG bindings for htm to avoid warnings on newer python
versions
Bug Fixes
- check for None header keywords when loading WCS objects
v0.6.4
-------------------------
New Features
- added pbar.PBar for simple progress bars
- make htm picklable (B. Van Klaveren)
Compatability
- Adapt to new pyyaml requirements to specify a loader, in io.read_yaml
(M. Becker)
- Updating setup.py for better macOS support (D. Munha)
v0.6.3.x
-------------------------
New Features
- ostools.StagedInFile
A class to stage a file in to local disk for reading.
- ostools.StagedOutFile
A context manager for staging files from temporary directories to
a final destination.
- added dorot option for dealing with random cap generation at the pole
- added coords.rotate to do euler rotations
- smart copying of Cosmo objects
- added pbar.PBar for simple progress bars
python 3 updates
- sqlite_util updated to py3
- sfile updated to py3
- exec_process updated to py3
numpy updates
- wcsutil, was still using old integer based ordering, now only
strings are supported
- deal properly with init numpy in recent versions
bug fixes
- fixed bug in makedirs_fromfile using wrong variable
- bug in fits writing, wrong variable name, also bug in verbosity
v0.6.2
-------------------------
Backward Incompatible Changes
- combine=True is now the default when reading lists of
files using esutil.io.read
Bug Fixes
- lots of py3 compatability stuff
- stat/util.py fix bug with data type in nperbin scheme
- plotting.py fake_points allow no color sent
- ostools.makedirs_fromfile
- wcsutil.py fix bug in _findxy for scalar input thanks to
Thomas Erben
v0.6.1
-------------------------
New Features
- numpy_util.dictlist2array
Convert a list of dictionaries to a structured numpy array. Works
for simple typs such as strings, integers, floats.
- stat.Binner
Now by default run calc_stats when calling dohist; I found I
essentially never did just dohist without calling calc_stats.
- ostools.makedirs_fromfile
new keyword allow_fail to allow failure
- io module
when fitsio is not available, look for astropy.io.fits as well
as pyfits (long overdue)
- stats.print_stats
Added nsigma= keyword
- random.Generator
Allow sending the random number generator through rng=
Bug Fixes
- plotting.compare_hist
Fix bug regarding binsize
v0.6.0 (2016-08-19)
---------------------
New Features
- python3 support
- unit tests. Coverage is not yet complete.
bug fixes
- in htm, fixed radius size check to match ra1/dec1 instead of ra2/dec2
(thanks to Eli Rykoff)
- some potential integer division issues
notes
- NumpyVector.h etc. do not seem to be reliable in python3. All use of
them has been removed from esutil, they are left for backwards
compatibility
v0.5.4 (2016-06-09)
---------------------
New:
- esutil.stats
- added print_stats() function, prints the stats generated by
get_stats(). Replaces doprint=True functionality of get_stats()
- esutil.numpy_util
- New faster match() code from Eli Rykoff
Updates
- NumpyVector.h use requirements = NPY_NOTSWAPPED | NPY_ENSUREARRAY;
- in esutil.wcs From Matt Becker, fixed roundoff error in wcs rotations;
added tolerance option to distortion inversions
Bug Fixes
- fixed bug in esutil.stat.get_stats not dealing properly with scalars
v0.5.3 (2015-11-10)
----------------------------------------------------------------------
New:
- htm:
- introduced the Matcher class, which is much more memory efficient for
matching than the old "reverse indices" style.
- Added intersect() method to the HTM class to look up all triangles
that are contained within or intersect a circle centered on the input
point.
- numpy_util.py:
- between: Test if array elements are within a range
- outside: Test if array elements are outside a range
- select_percentile(): select data in the given percentile(s)
- plotting.py
- added fake_points for getting symbols for a plot key. Useful when
the original points are not available or show up too small in the
key. fake_filled_circles now calls this function
- added Grid for simplifying working with a grid of plot
- added compare_hist function
- stats.py
- added get_stats() function to calculate stats for an array
- integrate
- added 2d gauss legendre integrator
- wcsutil
- added get_jacobian() function to calculate the jacobian
matrix of the transformation
- added get_naxis() method to get the dimensions, deals properly
with compressed fits extensions
- random
- added CholeskySampler
Updates:
- wsutil
- sped up by factor of 2.3 in some scenarios due to array
checking
- htm
- HTM class:
- Use 64-bit integer internally to allow higher depth values.
- match() uses a Matcher object internally
- stat.histogram2d
- Now uses proper index order for x,y
- plotting.py
- removed bcircle and bbox since these are actually part of biggles,
just not documented.
- Allow sizes= sent to fake_filled_circles
- stats
- wmom can work on arrays with more than one dimension
- weights can be sent to histogram2d
but fixes:
- coords.py
- fix units of returned radius
- added missing bounds checking
- fix bug in coords.dec_parse. The sign was not being used
- setup.py
- corrected deprecated use of include_dirs, now should work on all
systems
v0.5.2 (2014-06-12)
----------------------------------------------------------------------
Updates:
esutil/io.py:
Added writing of xml files.
Added bunzipping of fits files that end in .bz2
esutil/wcsutil.py:
Now allow any header object that supports iteration
esutil/plotting.py
- Allow sending norm for histogram plot function bhist.
- forward keywords to the underlying biggles routines
New methods:
esutil/numpy_util.py:
descr_to_native:
Remove byte order information from numpy type descriptor.
strmatch:
match a string array to a regular expression
match_multi:
code for matching two numpy integer arrays, one
of which may be non-unique. Similar to match_multi
from sdssidl (from Eli Rykoff)
rem_dup:
Return indices of unique values of an array, selecting the one
(when duplicates exist) with the largest value of flag. Optionally
returns the values in the array as well as their indices. (from Amy
Kimball)
esutil/random.py:
Added some new distributions
esutil/stats/util.py:
wmedian: calculate the weighted median
esutil/plotting.py:
get_grid Get a 2-d grid layout given the total number of plots
esutil/misc.py
collect_keyby:
Create a new dictionary from the input collection, keyed by the
values specified by the input key name.
Bug fixes
esutil/xmltools.py stderr used but not imported
esutil/plotting.py: deal properly with log plots when values
go below zero
esutil/coords.py:
bug fixes to coords.ra_parse and coords.dec_pars from Paul Ray and Dave
Smith, NRL
esutil/wcsutil.py
- allow scalar structured arrays in wcs
- use simple inversion if full distortion is not present
- don't use minimozer when distortion is none
- don't require npix and cunit1
v0.5.1 (2012-04-27)
----------------------------------------------------------------------
Updates:
Updated a number of places to use stderr for messages.
esutil/coords.py: Added randcap function to produce random
points in a spherical cap. Use clip() instead of
where in sphdist(), gcirc(). Added rec_area() to
calculate the area of a rectangle in longitude and
latitude.
esutil/hfds.py: Added "du" function to calculate disk usage.
Added is_in_hdfs() function. File listings now include
the hdfs:// in front by default.
esutil/io.py: In read() Added ability to read rec file with
no header of dtype is sent.
esutil.stat: Added cov2cor and cor2cov functions to convert
between covariance matrices and correlation matrices.
esutil/misc.py: Added wlog() for quick logging to stderr.
esutil/plotting.py: Added width keyword to bscatter.
v0.5.0 (2012-01-15)
----------------------------------------------------------------------
New files:
esutil/hdfs.py: routines to get files in and out of the hadoop
file system. Also routines for listing, removing, etc.
esutil/cosmology:
Now using simplified cosmology package written in C.
esutil/fits.py:
This is a workaround for reading subsets of columns and rows from fits
files when only pyfits is available. Note you should use
the fitsio package instead: https://github.com/esheldon/fitsio
The io module will select fitsio if available, otherwise pyfits.
esutil/sqlite_util.py:
Wrapper class and functions to read/write from sqlite.
Removed Files:
Removed patched pyfits. Trying to use the fitsio package
when available.
Updates:
General:
Removed some features that did not exist in python 2.5, such
as as format() functions for strings and importing print function
from future.
esutil/cosmology:
- The comoving volume is now for the whole sky. Previously it
was per steradian. To get the old behavior, divide by 4*pi
esutil/coords.py:
- added more optionals units to sphdist
esutil/plotting.py:
- bhist now doesn't use Histogram but a new histogram plotter. ylog
and xlog now work
- in plotrand use dot as default stuyle
- added fake filled circle point maker for when plotting dots and need
a key
- plotting.bscatter works better with log plots now
- added asinh_scale and norm to plotting, for images
- can send type= for curves in bscatter.
- now can use weights in plot hist.
esutil/io.py:
- If the filenames start with hdfs:// the files will be staged
in and out of the hadoop file system transparently.
- Use the fitsio package if available, fall back to pyfits otherwise.
https://github.com/esheldon/fitsio
- Added read_header function
- determine mode from input mode not from file object
- added reading and writing of yaml
- deal with file names that have no extensioin
- use **keys in read() and write()
- support write rec files.
esutil/numpy_util.py:
- Added sort option to numpy_util.random_subset
- added ability to print latex table to ArrayWriter and aprint
- ahelp can be used on scalars.
- added replicate function
- added arr2str function, which uses ArrayWriter
- added where1 convenience function, which uses w,=numpy.where() for 1D
arrays.
- replicate is now smarter, determining the data type from the input.
esutil/stat:
- Added Binner class to handle all histogramming, including
weighted histograms, etc.
- updated histogram to use binner
- added qgauss convenience function.
- added nbin= keyword for histograms
esutil/ostools.py:
- added makedirs_fromfile procedure; grabs the directory name
and creates it if it doesn't exist.
Bugs:
esutil/numpy_util.py:
- fixed bug with ensure_native
esutil/stat/util.py
- fixed bug sending nbin for histogram.
- Fix bug in stat.sigma_clip
- In Binner, make sure stats are fully calculated if weights are sent
- fixed bug in histogram when only min= or max= are sent
esutil/plotting.py:
- fixed bug for point type in plotrand.
esutil/cosmology:
- Use exact speed of light value.
esutil/io.py:
- fixed bug where file paths were not always being expanded
v0.4.1 (2010-08-18)
----------------------------------------------------------------------
New files:
esutil/include/NumpyRecords.h: A header only class for working
with record arrays in C++. (ESS)
esutil/include/NumpyVoidVector.h: A header only class for
working with numpy vectors in C++. Unlike NumpyVector.h,
which is a template class, the type is determined at
runtime.
esutil/algorithm.py: New module to hold algorithms. Currently
just playing with sorting algorithms.
Modifications:
esutil/numpy_util:
* Improvements to ArrayWriter, which is now used for
aprint() (ESS)
* Added random_subset() function.
esutil/htm/htm.py: cylmatch added (BG)
esutil/coords.py: Added shiftlon to shift a longitutde coordinate,
also shiftra which is an alias for shiftlon.
esutil/io.py: write() now supports json.
esutil/stomp_util.py: in_window() can run map.Contains() as well
as quadrant checking if radius is set (calls quad_check)
esutil/plotting.py: Added convenience function bscatter() which
uses biggles as plotting engine.
Other:
Moved exec_process from misc.py to ostools.py
----------------------------------------------------------------------
v0.4.0 (2010-04-23) Lots of major changes and additions.
Additions:
pyfitspatch:
A patched version of pyfits that allows writing binary tables from a
numpy array with fields, recarray, etc. with a single call. The io.py
sub-module uses this for input and output from FITS. The patches have
been sent to the pyfits people and when incorporated pyfitspatch will
be dropped.
numpy_util.ahelp:
Print out a formatted description of the input array. If the array
has fields, individual descriptions are printed for each field. This
is designed to be similar to help, struct, /str in IDL.
numpy_util.aprint:
Print fields from the array in columns.
numpy.randind
Return nrand random indices, with replacement, in the open
range [0,nmax)
misc.colprint():
Print a set of input sequences or arrays in columns.
ostools.DirStack: A simple directory stack like that supported by unix
shells.
io.write: Support writing files. Currently only FITS is supported.
Enhancements:
stat.histogram can now return statistics about the histogram and
binning of the data. If getdict=True, a dictionary is returned
with the 'hist' as well as the lower end of each bin 'low',
'high', 'center', 'mean', 'median', 'std', 'err'. If the
new keyword weights= is sent, the weighted versions are also
returned as 'wmean', 'wstd', etc.
Rearrangement:
stat is now a sub-package. The histogram code has now been written
as a C++ extension.
gauleg is not a C++ extension instead of a scipy.weave thing.
Bug Fixes:
Fix in NumpyVector.h to allow for the fact that int64_t and npy_int64
actually have different type ids on OS X. This affected the htm code.
Re-implemented the histogramming code as a C extension instead of
using scipy.weave. This circumvents the bug in weave for gcc >= 4.3.
Same for gauleg.
-----------------------------------------------
v0.3.0 (2010-04-04)
New files and packages:
htm package:
Added C++ code for working with the Heierarchical Triangular Mesh
(HTM). This is a code for breaking the sky into spherical triangles, a
quad tree, which makes for fast searching. Includes the class HTM with
methods such as lookup_id for finding the htm id of an ra,dec, match
for finding matches between lists of ra,dec and bincount for counting
pairs in radial bins, useful for calculating correlation functions.
integrate.py:
Tools for numerical integration. Currently has a class for performing
gauss-legendre integration.
plotting.py:
Tools for plotting with matplotlib.
setuplot() for setting up the plotting and establishing a plot
object.
set_mino0r_ticks() for setting up minor tick mark locations in
a smart way.
whiskers: Plot whiskers, or arrowless vectors.
random.py:
Contains a class for generating random points from an arbitrary
probability distribution.
coords.py:
astro_util.py is now coords.py, appropriate since all the code was
coordinate transformations.
Other Changes:
Numerous changes that I forgot to write into this release notes. Will be
more diligent in the future.