Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot apply lambda function to DataFrame with columns of type numpy.datetime64 #18573

Closed
meditativeape opened this issue Nov 30, 2017 · 2 comments · Fixed by #18577
Closed
Labels
Apply Apply, Aggregate, Transform, Map Duplicate Report Duplicate issue or pull request
Milestone

Comments

@meditativeape
Copy link

Code Sample, a copy-pastable example if possible

import pandas as pd
import numpy as np

ts1 = np.datetime64('2017-11-29T03:30')
ts2 = np.datetime64('2017-11-29T03:45')
d = {'number' : pd.Series([1., 2.]), 'string': pd.Series(['haha', 'wawa']), 'datetime64': pd.Series([ts1, ts2])}
df = pd.DataFrame(d)
df.apply(lambda row: (row.number, row.string), axis=1)

Problem description

Applying lambda function to a DataFrame with columns of type numpy.datetime64 throws an unexpected error: ValueError: Shape of passed values is (2, 2), indices imply (2, 3). The error message is not helpful, and I believe this error is thrown due to a bug. Dropping columns of type numpy.datetime64 fixes the issue.

Expected Output

0    (1.0, haha)
1    (2.0, wawa)
dtype: object

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.14.final.0
python-bits: 64
OS: Darwin
OS-release: 16.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.20.3
pytest: 3.2.1
pip: 9.0.1
setuptools: 36.5.0.post20170921
Cython: 0.26.1
numpy: 1.13.3
scipy: 0.19.1
xarray: None
IPython: 5.4.1
sphinx: 1.6.3
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.1.0
openpyxl: 2.4.8
xlrd: 1.1.0
xlwt: 1.2.0
xlsxwriter: 1.0.2
lxml: 4.1.0
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: 1.1.13
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None

@jreback jreback added Apply Apply, Aggregate, Transform, Map Duplicate Report Duplicate issue or pull request labels Nov 30, 2017
@jreback jreback added this to the No action milestone Nov 30, 2017
@jreback
Copy link
Contributor

jreback commented Nov 30, 2017

duplicate of #15628

.apply has to guess the return shape from your opaque passed function. This is not always possible.

idiomatically what you are doing is quite odd, simply do this.

In [43]: df[['number', 'string']]
Out[43]: 
   number string
0     1.0   haha
1     2.0   wawa

@meditativeape
Copy link
Author

Thanks for the suggestion, @jreback !

jorisvandenbossche pushed a commit that referenced this issue Feb 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform, Map Duplicate Report Duplicate issue or pull request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants