Skip to content

Commit

Permalink
* Upgrade presets for CPython 3.12.0, NumPy 1.26.0, SciPy 1.11.3
Browse files Browse the repository at this point in the history
  • Loading branch information
saudet committed Oct 17, 2023
1 parent 4ec3676 commit 5901ae8
Show file tree
Hide file tree
Showing 75 changed files with 2,203 additions and 1,201 deletions.
10 changes: 5 additions & 5 deletions .github/actions/deploy-ubuntu/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,18 +64,18 @@ runs:
export CODENAME=$UBUNTU_CODENAME
if [[ ! "$ARCH" == "amd64" ]]; then
# https://github.com/actions/runner-images/issues/4589
$SUDO apt-add-repository -y ppa:ondrej/php
$SUDO apt-get -y install ppa-purge
$SUDO ppa-purge -y ppa:ondrej/php
# https://github.com/actions/runner-images/issues/675
$SUDO gem install apt-spy2
$SUDO apt-spy2 check
$SUDO apt-spy2 fix --commit
$SUDO sed -i 's/azure\.//' /etc/apt/apt-mirrors.txt /etc/apt/sources.list
$SUDO cat /etc/apt/apt-mirrors.txt /etc/apt/sources.list
$SUDO apt-get update
# https://github.com/actions/runner-images/issues/4589
$SUDO apt-add-repository -y ppa:ondrej/php
$SUDO apt-get -y install ppa-purge
$SUDO ppa-purge -y ppa:ondrej/php
fi
if [[ "$ARCH" == "i386" ]]; then
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
* Refactor and improve presets for PyTorch ([pull #1360](https://github.com/bytedeco/javacpp-presets/pull/1360))
* Include `mkl_lapack.h` header file in presets for MKL ([issue #1388](https://github.com/bytedeco/javacpp-presets/issues/1388))
* Map new higher-level C++ API of Triton Inference Server ([pull #1361](https://github.com/bytedeco/javacpp-presets/pull/1361))
* Upgrade presets for OpenCV 4.8.1, DNNL 3.3, OpenBLAS 0.3.24, CPython 3.11.5, NumPy 1.25.2, SciPy 1.11.2, LLVM 17.0.1, Leptonica 1.83.1, Tesseract 5.3.3, TensorFlow Lite 2.14.0, Triton Inference Server 2.38.0, ONNX 1.14.1, ONNX Runtime 1.16.1, TVM 0.13.0, and their dependencies
* Upgrade presets for OpenCV 4.8.1, DNNL 3.3, OpenBLAS 0.3.24, CPython 3.12.0, NumPy 1.26.0, SciPy 1.11.3, LLVM 17.0.1, Leptonica 1.83.1, Tesseract 5.3.3, TensorFlow Lite 2.14.0, Triton Inference Server 2.38.0, ONNX 1.14.1, ONNX Runtime 1.16.1, TVM 0.13.0, and their dependencies

### June 6, 2023 version 1.5.9
* Virtualize `nvinfer1::IGpuAllocator` from TensorRT to allow customization ([pull #1367](https://github.com/bytedeco/javacpp-presets/pull/1367))
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,8 @@ Each child module in turn relies by default on the included [`cppbuild.sh` scrip
* CMINPACK 1.3.8 https://github.com/devernay/cminpack
* FFTW 3.3.10 http://www.fftw.org/download.html
* GSL 2.7 http://www.gnu.org/software/gsl/#downloading
* CPython 3.11.x https://www.python.org/downloads/
* NumPy 1.25.x https://github.com/numpy/numpy
* CPython 3.12.x https://www.python.org/downloads/
* NumPy 1.26.x https://github.com/numpy/numpy
* SciPy 1.11.x https://github.com/scipy/scipy
* Gym 0.26.x https://github.com/openai/gym
* LLVM 17.0.x http://llvm.org/releases/download.html
Expand Down
4 changes: 2 additions & 2 deletions cpython/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Introduction
------------
This directory contains the JavaCPP Presets module for:

* CPython 3.11.5 https://www.python.org/
* CPython 3.12.0 https://www.python.org/

Please refer to the parent README.md file for more detailed information about the JavaCPP Presets.

Expand Down Expand Up @@ -54,7 +54,7 @@ We can use [Maven 3](http://maven.apache.org/) to download and install automatic
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>cpython-platform</artifactId>
<version>3.11.5-1.5.10-SNAPSHOT</version>
<version>3.12.0-1.5.10-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
Expand Down
6 changes: 3 additions & 3 deletions cpython/cppbuild.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ if [[ -z "$PLATFORM" ]]; then
fi

OPENSSL=openssl-3.1.3
CPYTHON_VERSION=3.11.5
CPYTHON_VERSION=3.12.0
download https://www.openssl.org/source/$OPENSSL.tar.gz $OPENSSL.tar.gz
download https://www.python.org/ftp/python/$CPYTHON_VERSION/Python-$CPYTHON_VERSION.tgz Python-$CPYTHON_VERSION.tgz

Expand Down Expand Up @@ -148,7 +148,7 @@ case $PLATFORM in
cp -r Lib/* ../lib/
cp -r Include/* PC/pyconfig.h ../include/
unzip -o ../lib/ensurepip/_bundled/pip* -d ../lib/
unzip -o ../lib/ensurepip/_bundled/setuptools* -d ../lib/
# unzip -o ../lib/ensurepip/_bundled/setuptools* -d ../lib/
;;
windows-x86_64)
mkdir -p ../include ../lib ../libs ../bin
Expand All @@ -161,7 +161,7 @@ case $PLATFORM in
cp -r Lib/* ../lib/
cp -r Include/* PC/pyconfig.h ../include/
unzip -o ../lib/ensurepip/_bundled/pip* -d ../lib/
unzip -o ../lib/ensurepip/_bundled/setuptools* -d ../lib/
# unzip -o ../lib/ensurepip/_bundled/setuptools* -d ../lib/
;;
*)
echo "Error: Platform \"$PLATFORM\" is not supported"
Expand Down
2 changes: 1 addition & 1 deletion cpython/platform/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

<groupId>org.bytedeco</groupId>
<artifactId>cpython-platform</artifactId>
<version>3.11.5-${project.parent.version}</version>
<version>3.12.0-${project.parent.version}</version>
<name>JavaCPP Presets Platform for CPython</name>

<properties>
Expand Down
4 changes: 2 additions & 2 deletions cpython/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

<groupId>org.bytedeco</groupId>
<artifactId>cpython</artifactId>
<version>3.11.5-${project.parent.version}</version>
<version>3.12.0-${project.parent.version}</version>
<name>JavaCPP Presets for CPython</name>

<dependencies>
Expand All @@ -35,7 +35,7 @@
<configuration>
<includePaths>
<includePath>${basedir}/cppbuild/${javacpp.platform}/include/</includePath>
<includePath>${basedir}/cppbuild/${javacpp.platform}/include/python3.11/</includePath>
<includePath>${basedir}/cppbuild/${javacpp.platform}/include/python3.12/</includePath>
</includePaths>
<linkPaths>
<linkPath>${basedir}/cppbuild/${javacpp.platform}/lib/</linkPath>
Expand Down
2 changes: 1 addition & 1 deletion cpython/samples/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>cpython-platform</artifactId>
<version>3.11.5-1.5.10-SNAPSHOT</version>
<version>3.12.0-1.5.10-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

import static org.bytedeco.cpython.global.python.*;

// #endif


/* Stuff with no proper home (yet) */
Expand Down
35 changes: 35 additions & 0 deletions cpython/src/gen/java/org/bytedeco/cpython/PerfMapState.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Targeted by JavaCPP version 1.5.10-SNAPSHOT: DO NOT EDIT THIS FILE

package org.bytedeco.cpython;

import java.nio.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.annotation.*;

import static org.bytedeco.javacpp.presets.javacpp.*;

import static org.bytedeco.cpython.global.python.*;


// #if !defined(Py_LIMITED_API)
@Properties(inherit = org.bytedeco.cpython.presets.python.class)
public class PerfMapState extends Pointer {
static { Loader.load(); }
/** Default native constructor. */
public PerfMapState() { super((Pointer)null); allocate(); }
/** Native array allocator. Access with {@link Pointer#position(long)}. */
public PerfMapState(long size) { super((Pointer)null); allocateArray(size); }
/** Pointer cast constructor. Invokes {@link Pointer#Pointer(Pointer)}. */
public PerfMapState(Pointer p) { super(p); }
private native void allocate();
private native void allocateArray(long size);
@Override public PerfMapState position(long position) {
return (PerfMapState)super.position(position);
}
@Override public PerfMapState getPointer(long i) {
return new PerfMapState((Pointer)this).offsetAddress(i);
}

public native @Cast("FILE*") Pointer perf_map(); public native PerfMapState perf_map(Pointer setter);
public native PyThread_type_lock map_lock(); public native PerfMapState map_lock(PyThread_type_lock setter);
}
65 changes: 13 additions & 52 deletions cpython/src/gen/java/org/bytedeco/cpython/PyASCIIObject.java
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

/* ASCII-only strings created through PyUnicode_New use the PyASCIIObject
structure. state.ascii and state.compact are set, and the data
immediately follow the structure. utf8_length and wstr_length can be found
immediately follow the structure. utf8_length can be found
in the length field; the utf8 pointer is equal to the data pointer. */
@Properties(inherit = org.bytedeco.cpython.presets.python.class)
public class PyASCIIObject extends Pointer {
Expand Down Expand Up @@ -44,8 +44,7 @@ public class PyASCIIObject extends Pointer {
* kind = PyUnicode_1BYTE_KIND
* compact = 1
* ascii = 1
* ready = 1
* (length is the length of the utf8 and wstr strings)
* (length is the length of the utf8)
* (data starts just after the structure)
* (since ASCII is decoded from UTF-8, the utf8 string are the data)
Expand All @@ -56,77 +55,45 @@ public class PyASCIIObject extends Pointer {
* kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or
PyUnicode_4BYTE_KIND
* compact = 1
* ready = 1
* ascii = 0
* utf8 is not shared with data
* utf8_length = 0 if utf8 is NULL
* wstr is shared with data and wstr_length=length
if kind=PyUnicode_2BYTE_KIND and sizeof(wchar_t)=2
or if kind=PyUnicode_4BYTE_KIND and sizeof(wchar_t)=4
* wstr_length = 0 if wstr is NULL
* (data starts just after the structure)
- legacy string, not ready:
* structure = PyUnicodeObject
* test: kind == PyUnicode_WCHAR_KIND
* length = 0 (use wstr_length)
* hash = -1
* kind = PyUnicode_WCHAR_KIND
* compact = 0
* ascii = 0
* ready = 0
* interned = SSTATE_NOT_INTERNED
* wstr is not NULL
* data.any is NULL
* utf8 is NULL
* utf8_length = 0
- legacy string, ready:
- legacy string:
* structure = PyUnicodeObject structure
* test: !PyUnicode_IS_COMPACT(op) && kind != PyUnicode_WCHAR_KIND
* test: !PyUnicode_IS_COMPACT(op)
* kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or
PyUnicode_4BYTE_KIND
* compact = 0
* ready = 1
* data.any is not NULL
* utf8 is shared and utf8_length = length with data.any if ascii = 1
* utf8_length = 0 if utf8 is NULL
* wstr is shared with data.any and wstr_length = length
if kind=PyUnicode_2BYTE_KIND and sizeof(wchar_t)=2
or if kind=PyUnicode_4BYTE_KIND and sizeof(wchar_4)=4
* wstr_length = 0 if wstr is NULL
Compact strings use only one memory block (structure + characters),
whereas legacy strings use one block for the structure and one block
for characters.
Legacy strings are created by PyUnicode_FromUnicode() and
PyUnicode_FromStringAndSize(NULL, size) functions. They become ready
when PyUnicode_READY() is called.
Legacy strings are created by subclasses of Unicode.
See also _PyUnicode_CheckConsistency().
*/
public native @ByRef PyObject ob_base(); public native PyASCIIObject ob_base(PyObject setter);
public native @Cast("Py_ssize_t") long length(); public native PyASCIIObject length(long setter); /* Number of code points in the string */
public native @Cast("Py_hash_t") long hash(); public native PyASCIIObject hash(long setter); /* Hash value; -1 if not set */
/*
SSTATE_NOT_INTERNED (0)
SSTATE_INTERNED_MORTAL (1)
SSTATE_INTERNED_IMMORTAL (2)
If interned != SSTATE_NOT_INTERNED, the two references from the
/* If interned is non-zero, the two references from the
dictionary to this object are *not* counted in ob_refcnt.
*/
The possible values here are:
0: Not Interned
1: Interned
2: Interned and Immortal
3: Interned, Immortal, and Static
This categorization allows the runtime to determine the right
cleanup mechanism at runtime shutdown. */
@Name("state.interned") public native @Cast("unsigned int") @NoOffset int state_interned(); public native PyASCIIObject state_interned(int setter);
/* Character size:
- PyUnicode_WCHAR_KIND (0):
* character type = wchar_t (16 or 32 bits, depending on the
platform)
- PyUnicode_1BYTE_KIND (1):
* character type = Py_UCS1 (8 bits, unsigned)
Expand Down Expand Up @@ -157,13 +124,7 @@ See also _PyUnicode_CheckConsistency().
and the kind is PyUnicode_1BYTE_KIND. If ascii is set and compact is
set, use the PyASCIIObject structure. */
@Name("state.ascii") public native @Cast("unsigned int") @NoOffset int state_ascii(); public native PyASCIIObject state_ascii(int setter);
/* The ready flag indicates whether the object layout is initialized
completely. This means that this is either a compact object, or
the data pointer is filled out. The bit is redundant, and helps
to minimize the test in PyUnicode_IS_READY(). */
@Name("state.ready") public native @Cast("unsigned int") @NoOffset int state_ready(); public native PyASCIIObject state_ready(int setter);
/* Padding to ensure that PyUnicode_DATA() is always aligned to
4 bytes (see issue #19537 on m68k). */

public native @Cast("wchar_t*") Pointer wstr(); public native PyASCIIObject wstr(Pointer setter); /* wchar_t representation (null-terminated) */
}
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,6 @@ public class PyAsyncGenObject extends Pointer {
}

public native @ByRef PyObject ob_base(); public native PyAsyncGenObject ob_base(PyObject setter);
/* The code object backing the generator */
public native PyCodeObject ag_code(); public native PyAsyncGenObject ag_code(PyCodeObject setter);
/* List of weak reference. */
public native PyObject ag_weakreflist(); public native PyAsyncGenObject ag_weakreflist(PyObject setter);
/* Name of the generator. */
Expand All @@ -50,4 +48,5 @@ public class PyAsyncGenObject extends Pointer {
public native byte ag_frame_state(); public native PyAsyncGenObject ag_frame_state(byte setter);
public native PyObject ag_iframe(int i); public native PyAsyncGenObject ag_iframe(int i, PyObject setter);
@MemberGetter public native @Cast("PyObject**") PointerPointer ag_iframe();

}
13 changes: 6 additions & 7 deletions cpython/src/gen/java/org/bytedeco/cpython/PyCodeObject.java
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,6 @@ public class PyCodeObject extends Pointer {
public native PyObject co_exceptiontable(); public native PyCodeObject co_exceptiontable(PyObject setter); /* Byte string encoding exception handling \
table */
public native int co_flags(); public native PyCodeObject co_flags(int setter); /* CO_..., see below */
public native short co_warmup(); public native PyCodeObject co_warmup(short setter); /* Warmup counter for quickening */
public native short _co_linearray_entry_size(); public native PyCodeObject _co_linearray_entry_size(short setter); /* Size of each entry in _co_linearray */

/* The rest are not so impactful on performance. */
public native int co_argcount(); public native PyCodeObject co_argcount(int setter); /* #arguments, except *args */
Expand All @@ -70,12 +68,12 @@ public class PyCodeObject extends Pointer {

/* redundant values (derived from co_localsplusnames and \
co_localspluskinds) */
public native int co_nlocalsplus(); public native PyCodeObject co_nlocalsplus(int setter); /* number of local + cell + free variables \
*/
public native int co_nlocalsplus(); public native PyCodeObject co_nlocalsplus(int setter); /* number of local + cell + free variables */
public native int co_framesize(); public native PyCodeObject co_framesize(int setter); /* Size of frame in words */
public native int co_nlocals(); public native PyCodeObject co_nlocals(int setter); /* number of local variables */
public native int co_nplaincellvars(); public native PyCodeObject co_nplaincellvars(int setter); /* number of non-arg cell variables */
public native int co_ncellvars(); public native PyCodeObject co_ncellvars(int setter); /* total number of cell variables */
public native int co_nfreevars(); public native PyCodeObject co_nfreevars(int setter); /* number of free variables */
public native @Cast("uint32_t") int co_version(); public native PyCodeObject co_version(int setter); /* version number */

public native PyObject co_localsplusnames(); public native PyCodeObject co_localsplusnames(PyObject setter); /* tuple mapping offsets to names */
public native PyObject co_localspluskinds(); public native PyCodeObject co_localspluskinds(PyObject setter); /* Bytes mapping to local kinds (one byte \
Expand All @@ -85,8 +83,9 @@ public class PyCodeObject extends Pointer {
public native PyObject co_qualname(); public native PyCodeObject co_qualname(PyObject setter); /* unicode (qualname, for reference) */
public native PyObject co_linetable(); public native PyCodeObject co_linetable(PyObject setter); /* bytes object that holds location info */
public native PyObject co_weakreflist(); public native PyCodeObject co_weakreflist(PyObject setter); /* to support weakrefs to code objects */
public native PyObject _co_code(); public native PyCodeObject _co_code(PyObject setter); /* cached co_code object/attribute */
public native @Cast("char*") BytePointer _co_linearray(); public native PyCodeObject _co_linearray(BytePointer setter); /* array of line offsets */
public native _PyCoCached _co_cached(); public native PyCodeObject _co_cached(_PyCoCached setter); /* cached co_* attributes */
public native @Cast("uint64_t") long _co_instrumentation_version(); public native PyCodeObject _co_instrumentation_version(long setter); /* current instrumentation version */
public native _PyCoMonitoringData _co_monitoring(); public native PyCodeObject _co_monitoring(_PyCoMonitoringData setter); /* Monitoring data */
public native int _co_firsttraceable(); public native PyCodeObject _co_firsttraceable(int setter); /* index of first traceable instruction */
/* Scratch space for extra data relating to the code object. \
Type is a void* to keep the format private in codeobject.c to force \
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Targeted by JavaCPP version 1.5.10-SNAPSHOT: DO NOT EDIT THIS FILE

package org.bytedeco.cpython;

import java.nio.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.annotation.*;

import static org.bytedeco.javacpp.presets.javacpp.*;

import static org.bytedeco.cpython.global.python.*;



/*
* A callback that is invoked for different events in a code object's lifecycle.
*
* The callback is invoked with a borrowed reference to co, after it is
* created and before it is destroyed.
*
* If the callback sets an exception, it must return -1. Otherwise
* it should return 0.
*/
@Properties(inherit = org.bytedeco.cpython.presets.python.class)
public class PyCode_WatchCallback extends FunctionPointer {
static { Loader.load(); }
/** Pointer cast constructor. Invokes {@link Pointer#Pointer(Pointer)}. */
public PyCode_WatchCallback(Pointer p) { super(p); }
protected PyCode_WatchCallback() { allocate(); }
private native void allocate();
public native int call(
@Cast("PyCodeEvent") int event,
PyCodeObject co);
}
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,4 @@ public class PyCompactUnicodeObject extends Pointer {
public native @Cast("Py_ssize_t") long utf8_length(); public native PyCompactUnicodeObject utf8_length(long setter); /* Number of bytes in utf8, excluding the
* terminating \0. */
public native @Cast("char*") BytePointer utf8(); public native PyCompactUnicodeObject utf8(BytePointer setter); /* UTF-8 representation (null-terminated) */
public native @Cast("Py_ssize_t") long wstr_length(); public native PyCompactUnicodeObject wstr_length(long setter); /* Number of code points in wstr, possible
* surrogates count as two code points. */
}
Loading

0 comments on commit 5901ae8

Please sign in to comment.