Numpy-> Cython conversione: Errore di compilazione: Impossibile convertire 'npy_intp *' per oggetto Python

Question

Ho il seguente codice che deve essere propperly convertito in Cython:

from numpy import * 

## returns winning players or [] if undecided. 
def score(board): 
    scores = [] 
    checked = zeros(board.shape) 
    for i in xrange(len(board)): 
     for j in xrange(len(board)): 
      if checked[i,j] == 0 and board[i,j] !=0: 
       ... do stuf 

il mio tentativo di conversione al Cython:

import numpy as np 
cimport numpy as np 

@cython.boundscheck(False) 
@cython.wraparound(False) 
@cython.nonecheck(False) 

## returns winning players or [] if undecided. 
def score(np.ndarray[int, ndim=2] board): 
    scores = [] 
    cdef np.ndarray[int, ndim = 2 ] checked 
    checked = np.zeros(board.shape) 
    for i in xrange(len(board)): 
     for j in xrange(len(board)): 
      if checked[i,j] == 0 and board[i,j] !=0: 
       ... do stuf 

Ma quando compilo ottengo:

$ python setup.py build_ext --inplace 
running build_ext 
cythoning newgox.pyx to newgox.c 


## returns winning players or [] if undecided. 
def score(np.ndarray[int, ndim=2] board): 
    scores = [] 
    cdef np.ndarray[int, ndim = 2 ] checked 
    checked = np.zeros(board.shape) 
        ^
------------------------------------------------------------ 

newgox.pyx:58:28: Cannot convert 'npy_intp *' to Python object 
building 'newgox' extension 

Inoltre, io non sono sicuro che questo è il modo giusto di lavorare con lis ts in Cython:

scores = [] 
if some_stuff_is_true: 
    scores.append(some_integer) 

EDIT:

Grazie, il codice ora compila ma quando l'eseguo ottengo l'errore:

File "newgox.pyx", line 63, in newgox.score (newgox.c:1710) 
    def score(np.ndarray[np.int, ndim=2] board): 
ValueError: Buffer dtype mismatch, expected 'int object' but got 'long' 

ho legato queste due opzioni:

ctypedef np.int_t DTYPE_t 
DTYPE = np.int 

e quindi continuare su:

board = zeros((5,5), dtype = DTYPE) 
def score(np.ndarray[DTYPE, ndim=2] board): 

o semplicemente in entrambe le dichiarazioni:

... np.int ... 

Il risultato è lo stesso errore, ma con la firma:

ValueError: Buffer dtype mismatch, expected 'int' but got 'long' 

---

Grazie bellamyj, ma il codice suggerito non viene compilato,

$ python setup.py build_ext --inplace 
running build_ext 
cythoning newgox.pyx to newgox.c 
building 'newgox' extension 
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c newgox.c -o build/temp.macosx-10.6-intel-2.7/newgox.o 

newgox.c:238:31: error: numpy/arrayobject.h: No such file or directory 
newgox.c:239:31: error: numpy/ufuncobject.h: No such file or directory 
newgox.c:356: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__pyx_t_5numpy_int8_t’ 
newgox.c:365: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__pyx_t_5numpy_int16_t’ 
newgox.c:374: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__pyx_t_5numpy_int32_t’ 

E continua t o enumerare anche tutti gli altri tipi.

allora mi dice questo:

newgox.c:978: error: expected ‘)’ before ‘*’ token 
newgox.c:979: error: expected ‘)’ before ‘*’ token 
newgox.c:980: error: expected ‘)’ before ‘*’ token 
newgox.c:983: error: expected declaration specifiers or ‘...’ before ‘PyArrayObject’ 
newgox.c:984: error: expected declaration specifiers or ‘...’ before ‘PyArrayObject’ 
newgox.c:985: error: expected ‘)’ before ‘*’ token 
newgox.c:987: error: ‘__pyx_t_5numpy_int32_t’ undeclared here (not in a function) 
newgox.c: In function ‘__pyx_pf_6newgox_7score’: 
newgox.c:1638: error: ‘PyArrayObject’ undeclared (first use in this function) 
newgox.c:1638: error: (Each undeclared identifier is reported only once 
newgox.c:1638: error: for each function it appears in.) 
newgox.c:1638: error: ‘__pyx_v_checked’ undeclared (first use in this function) 
newgox.c:1659: error: ‘__pyx_t_5’ undeclared (first use in this function) 
newgox.c:1721: error: expected expression before ‘)’ token 
newgox.c:1723: error: expected expression before ‘)’ token 
newgox.c:1747: error: expected expression before ‘)’ token 
newgox.c:1766: error: expected expression before ‘)’ token 
newgox.c:1813: error: expected expression before ‘)’ token 
newgox.c:1846: error: expected expression before ‘)’ token 
newgox.c:1846: error: too many arguments to function ‘__pyx_f_6newgox_check_life’ 
newgox.c:2009: error: expected expression before ‘)’ token 
newgox.c:2009: warning: assignment makes pointer from integer without a cast 
newgox.c:2012: error: expected expression before ‘)’ token 
newgox.c:2032: error: expected expression before ‘)’ token 
newgox.c: At top level: 
newgox.c:2088: error: expected declaration specifiers or ‘...’ before ‘PyArrayObject’ 
newgox.c: In function ‘__pyx_f_6newgox_check_life’: 
newgox.c:2124: error: ‘__pyx_v_board’ undeclared (first use in this function) 
newgox.c:2160: error: ‘PyArrayObject’ undeclared (first use in this function) 
newgox.c:2160: error: expected expression before ‘)’ token 
newgox.c:2160: error: too many arguments to function ‘__pyx_f_6newgox_liberty’ 
newgox.c:2420: error: too many arguments to function ‘__pyx_f_6newgox_check_life’ 
newgox.c: At top level: 
newgox.c:2583: error: expected declaration specifiers or ‘...’ before ‘PyArrayObject’ 
newgox.c: In function ‘__pyx_f_6newgox_liberty’: 
newgox.c:2610: error: ‘__pyx_v_board’ undeclared (first use in this function) 
newgox.c: At top level: 
newgox.c:2859: error: expected ‘)’ before ‘*’ token 
newgox.c: In function ‘__pyx_pf_5numpy_7ndarray___getbuffer__’: 
newgox.c:2999: error: ‘PyArray_Descr’ undeclared (first use in this function) 
newgox.c:2999: error: ‘__pyx_v_descr’ undeclared (first use in this function) 
newgox.c:3062: error: ‘PyArrayObject’ undeclared (first use in this function) 
newgox.c:3062: error: expected expression before ‘)’ token 
newgox.c:3071: error: ‘npy_intp’ undeclared (first use in this function) 
newgox.c:3114: error: expected expression before ‘)’ token 
newgox.c:3114: error: ‘NPY_C_CONTIGUOUS’ undeclared (first use in this function) 
newgox.c:3154: error: expected expression before ‘)’ token 
newgox.c:3154: error: ‘NPY_F_CONTIGUOUS’ undeclared (first use in this function) 
newgox.c:3184: error: expected expression before ‘)’ token 
newgox.c:3184: warning: assignment makes pointer from integer without a cast 
newgox.c:3240: error: expected expression before ‘)’ token 
newgox.c:3240: error: subscripted value is neither array nor pointer 
newgox.c:3249: error: expected expression before ‘)’ token 
newgox.c:3249: error: subscripted value is neither array nor pointer 
newgox.c:3262: error: expected expression before ‘)’ token 
newgox.c:3271: error: expected expression before ‘)’ token 
newgox.c:3291: error: expected expression before ‘)’ token 

E ancora un po '.

Avviso dice:

newgox.c:2160: error: too many arguments to function ‘__pyx_f_6newgox_liberty’ 
newgox.c:2420: error: too many arguments to function ‘__pyx_f_6newgox_check_life’ 

Ma la mia chiamata dal punteggio/1 a check_life è:

life, newly_checked = check_life(i,j,board,[]) 

Ed ricevuto in questo modo:

# helper functions of score/1 
cdef check_life(int i, int j, np.ndarray[int, ndim=2] board, checked): 
    ... 
    return life, checked 

Infine, ciò che tipo di dati è il "i4" che usi?

Answer 1

Non penso che la decompressione tuple funzioni in Cython come in Python. È necessario specificare la dimensione di checked utilizzando (board.shape[0], board.shape[1]).

È inoltre necessario specificare il tipo di dati di checked. Per impostazione predefinita, np.zeros restituisce un float64. È necessario modificarlo in un int32 per essere compatibile con il tipo dichiarato.

Ecco una versione modificata da eseguire.

def score(np.ndarray[int, ndim=2] board): 

    cdef np.ndarray[np.int32_t, ndim = 2] checked 
    checked = np.zeros((board.shape[0], board.shape[1]), dtype='i4') 

    for i in xrange(len(board)): 
     for j in xrange(len(board)): 
      if checked[i,j] == 0 and board[i,j] !=0: 
       ... do stuff 

Per rispondere alla seconda parte della domanda, è possibile aggiungere alle liste le funzioni di Cython. Comunque, a quanto ho capito, questo comporta chiamate di funzione Python, quindi non sarà più veloce delle stesse operazioni in Python.

Update 1:

credo quegli errori sono perché il compilatore non riesce a trovare i file header numpy. Aggiorna il tuo setup.py per includere il percorso di quei file. Il percorso dovrebbe essere emesso dalla funzione np.get_include().

import numpy as np 
from distutils.core import setup 
from distutils.extension import Extension 
from Cython.Distutils import build_ext 

setup(
     cmdclass={'build_ext': build_ext}, 
     ext_modules = [ 
       Extension("new_gox", 
         ["new_gox.pyx"], 
         include_dirs=[np.get_include()]) 
        ] 
    ) 

Il dtype i4 è un intero a 32 bit. È equivalente a np.int32.

Non sono sicuro di cosa succede con gli errori "troppi argomenti". Se chiami quella funzione da Python, deve essere dichiarata come def o cpdef. Le funzioni dichiarate con cdef possono essere chiamate solo da Cython. Questo è descritto nello here nei documenti Cython.

Answer 2

np.intp è uguale a (void *)

definire il suo tipo come questo:

ctypedef np.npy_intp SIZE_t    # Type for indices and counters 

fonte:

NPY_INTP == The enumeration value for a signed integer type which is the same size as a (void *) pointer. This is the type used by all arrays of indices.

Così, per esempio in Cython ...

Struct nell'intestazione file.h file:

struct Cudamat_Struct1 { 
    float * dataHost; 
    float * dataDevice; 
    int onDevice; 
    int onHost; 
    (void *) shape; 
    int size; 
    int isTrans; // 0 or 1 
    int ownsData; 

}; 

In Cython file di file.pxd:

cdef extern from "file.h": 
    cdef cppclass c_Cudamat_Struct "Cudamat_Struct": 
     float *dataHost 
     float *dataDevice 
     bint onDevice 
     bint onHost 
     SIZE_t * shape 
     int size 
     bint isTrans 
     bint ownsData 

Infine nella funzione Cython:

cdef pycublas_thrust_2d(): 
    cdef c_Cudamat_Struct1* mat = new c_Cudamat_Struct1() 

    cdef int N = 10 

    cdef np.ndarray[ndim=2, dtype=F32t] arrHost = np.ones((1,10),dtype=F32) 

    cdef int size = 10 

    cdef F32t[:,:] arrHost_ = arrHost 

    cdef F32t * arrHostPtr = &arrHost[0,0] 

    mat.dataHost = arrHostPtr 
    mat.shape = arrHost.shape 
    mat.size = size