ENH: Add Index.to_frame method #17815

gfyoung · 2017-10-08T01:58:02Z

Title is self-explanatory. Closes #15230.

codecov · 2017-10-08T02:30:29Z

Codecov Report

Merging #17815 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #17815      +/-   ##
==========================================
- Coverage   91.26%   91.24%   -0.02%     
==========================================
  Files         163      163              
  Lines       49978    49990      +12     
==========================================
+ Hits        45611    45614       +3     
- Misses       4367     4376       +9

Flag	Coverage Δ
#multiple	`89.05% <100%> (ø)`	⬆️
#single	`40.24% <16.66%> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/indexes/base.py	`96.47% <100%> (+0.01%)`	⬆️
pandas/core/indexes/datetimes.py	`95.61% <100%> (+0.02%)`	⬆️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.74% <0%> (-0.1%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7db7f82...75f11cc. Read the comment docs.

codecov · 2017-10-08T02:30:32Z

Codecov Report

Merging #17815 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #17815      +/-   ##
==========================================
- Coverage   91.26%   91.24%   -0.02%     
==========================================
  Files         163      163              
  Lines       49980    49992      +12     
==========================================
+ Hits        45613    45616       +3     
- Misses       4367     4376       +9

Flag	Coverage Δ
#multiple	`89.05% <100%> (ø)`	⬆️
#single	`40.24% <16.66%> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/indexes/multi.py	`96.39% <ø> (ø)`	⬆️
pandas/core/indexes/datetimes.py	`95.61% <100%> (+0.02%)`	⬆️
pandas/core/indexes/base.py	`96.47% <100%> (+0.01%)`	⬆️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.74% <0%> (-0.1%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f0ee53...1c0aaec. Read the comment docs.

jschendel · 2017-10-08T02:53:13Z

pandas/core/indexes/base.py

@@ -1005,6 +1005,29 @@ def to_series(self, **kwargs):
                      index=self._shallow_copy(),
                      name=self.name)

+    def to_frame(self, index=True):
+        """
+        Create a DataFrame with the columns the levels of the Index


I think this could be reworded. Unless I'm missing something this should only return one column for the base Index, which should only have a single level (MultiIndex has it's own method/docstring). So maybe something like "Create a DataFrame with the Index values as a column".

Yeah...bad English copied and pasted. Fixed.

jschendel · 2017-10-08T02:53:23Z

pandas/core/indexes/base.py

+        Parameters
+        ----------
+        index : boolean, default True
+            return the Index as the index


This was a little confusing to me on first read. Maybe something along the lines of "use this Index as the index of the resulting DataFrame" would be a little more clear?

Yeah...bad English copied and pasted. Fixed.

jschendel · 2017-10-08T02:53:58Z

pandas/core/indexes/datetimes.py

@@ -913,6 +913,45 @@ def to_series(self, keep_tz=False):
                      index=self._shallow_copy(),
                      name=self.name)

+    def to_frame(self, index=True, keep_tz=False):
+        """
+        Create a DataFrame with the columns the levels of the Index


Yeah...bad English copied and pasted. Fixed.

jschendel · 2017-10-08T02:54:09Z

pandas/core/indexes/datetimes.py

+        Parameters
+        ----------
+        index : boolean, default True
+            return the DatetimeIndex as the index


Yeah...bad English copied and pasted. Fixed.

jschendel · 2017-10-08T02:54:16Z

pandas/core/indexes/datetimes.py

+
+        Returns
+        -------
+        DataFrame : a DataFrame containing the original Index data.


"Index" -> "DatetimeIndex"

jreback · 2017-10-08T16:24:00Z

pandas/tests/indexes/common.py

@@ -51,6 +51,20 @@ def test_to_series(self):
        assert s.index is not idx
        assert s.name == idx.name

+    def test_to_frame(self):
+        idx = self.create_index()


add the issue number

jreback · 2017-10-08T16:25:22Z

pandas/core/indexes/base.py

+        DataFrame : a DataFrame containing the original Index data.
+        """
+
+        from pandas import DataFrame


this does not do the same thing as .to_series() where the values and the index are the same.
is there a reason you are doing this?

where the values and the index are the same.

Not sure I get you here. The implementation I wrote tries to be consistent with what was done with MultiIndex by constructing DataFrame with data and setting the index if needed.

ok this is reasonable, side issue this is a little suspect:

In [4]: pd.MultiIndex.from_product([range(3),list('ab')], names=['foo', 'bar']).to_frame() Out[4]: bar foo foo bar 0 a a 0 b b 0 1 a a 1 b b 1 2 a a 2 b b 2 In [5]: pd.MultiIndex.from_product([range(3),list('ab')], names=['foo', 'bar']).to_series() Out[5]: foo bar 0 a (0, a) b (0, b) 1 a (1, a) b (1, b) 2 a (2, a) b (2, b) dtype: object

What is suspect about it?
The index seems toe same in both examples?

gfyoung · 2017-10-08T23:41:43Z

@jreback @jschendel : Addressed all comments, and everything is green. PTAL.

jschendel · 2017-10-09T01:21:45Z

pandas/core/indexes/multi.py

@@ -1010,14 +1010,14 @@ def _to_safe_for_reshape(self):

    def to_frame(self, index=True):
        """
-        Create a DataFrame with the columns the levels of the MultiIndex
+        Create a DataFrame with the columns and levels of the MultiIndex.


This doesn't seem quite right. Maybe something like "Create a DataFrame with the levels of the MultiIndex as columns"?

Sure, done.

jschendel · 2017-10-09T01:22:28Z

Minor comment, otherwise LGTM.

jorisvandenbossche

Added some comments!

(I don't feel too enthusiastic about adding this method, but given the consistency between Index and MultiIndex I suppose this is good)

jorisvandenbossche · 2017-10-09T07:19:13Z

doc/source/whatsnew/v0.21.0.txt

@@ -31,6 +31,7 @@ New features
 - Added ``skipna`` parameter to :func:`~pandas.api.types.infer_dtype` to
  support type inference in the presence of missing values (:issue:`17059`).
 - :class:`~pandas.Resampler.nearest` is added to support nearest-neighbor upsampling (:issue:`17496`).
+- :class:`~pandas.Index~` has added support for a ``to_frame`` method (:issue:`15230`)


The second ~ should not be there I think

jorisvandenbossche · 2017-10-09T07:24:12Z

pandas/core/indexes/datetimes.py

+            Set the index of the returned DataFrame as self.
+
+        keep_tz : optional, defaults False.
+            return the data keeping the timezone.


Is there a reason to have this keyword?
I suppose the reason is for consistency with to_series, but that has this keyword for historical reasons (I think): it dates from before we could have tz aware data inside a series / frame.

Given that, I am not sure it is needed to copy that here.

@jorisvandenbossche : Given that you're not 100% sure with this logic, I'd rather stick with consistency. We can always remove (and deprecate) the argument in one sweep.

If we only think of deprecating it, then it makes no sense to add it IMO.
(you can say the same with "we can always add it later if people want it" :-))

jorisvandenbossche · 2017-10-09T07:25:40Z

pandas/core/indexes/multi.py


        .. versionadded:: 0.20.0

        Parameters
        ----------
        index : boolean, default True
-            return this MultiIndex as the index
+            Set the index of the returned DataFrame as self.


I personally would not use "self" in docstrings. For users that are not very familiar with writing classes themselves (and you don't need to do this for using pandas) this will sound very strange.

Fair enough. Fixed.

jorisvandenbossche · 2017-10-09T07:27:02Z

pandas/tests/indexes/common.py

+    def test_to_frame(self):
+        # see gh-15230
+        idx = self.create_index()
+        name = idx.name or 0


There is at least one in the create_index indices that has a name? (or at least one that has no name)

Yes, there is.

Closes pandas-devgh-15230.

jreback · 2017-10-09T12:17:33Z

thanks. maybe make an issue about this #17815 (comment) if you would

Closes pandas-devgh-15230.

gfyoung added Enhancement Indexing Related to indexing on series/frames, not to indexes themselves labels Oct 8, 2017

gfyoung added this to the 0.21.0 milestone Oct 8, 2017

jschendel reviewed Oct 8, 2017

View reviewed changes

jreback requested changes Oct 8, 2017

View reviewed changes

jreback removed this from the 0.21.0 milestone Oct 8, 2017

gfyoung force-pushed the to-frame-index branch from 75f11cc to 88d3eed Compare October 8, 2017 20:48

jschendel reviewed Oct 9, 2017

View reviewed changes

gfyoung force-pushed the to-frame-index branch from 88d3eed to 554ecd0 Compare October 9, 2017 01:32

jorisvandenbossche reviewed Oct 9, 2017

View reviewed changes

pandas-dev deleted a comment from jorisvandenbossche Oct 9, 2017

gfyoung force-pushed the to-frame-index branch from 554ecd0 to 587dc1d Compare October 9, 2017 07:36

ENH: Add Index.to_frame method

1c0aaec

Closes pandas-devgh-15230.

gfyoung force-pushed the to-frame-index branch from 587dc1d to 1c0aaec Compare October 9, 2017 07:38

jreback added this to the 0.21.0 milestone Oct 9, 2017

jreback approved these changes Oct 9, 2017

View reviewed changes

jreback merged commit 2ebe085 into pandas-dev:master Oct 9, 2017

jorisvandenbossche mentioned this pull request Oct 9, 2017

Remove keep_tz kwarg from DatetimeIndex.to_frame #17826

Merged

gfyoung deleted the to-frame-index branch October 9, 2017 16:34

ghost pushed a commit to reef-technologies/pandas that referenced this pull request Oct 16, 2017

ENH: Add Index.to_frame method (pandas-dev#17815)

2075a8c

Closes pandas-devgh-15230.

alanbato pushed a commit to alanbato/pandas that referenced this pull request Nov 10, 2017

ENH: Add Index.to_frame method (pandas-dev#17815)

bddd7e7

Closes pandas-devgh-15230.

No-Stream pushed a commit to No-Stream/pandas that referenced this pull request Nov 28, 2017

ENH: Add Index.to_frame method (pandas-dev#17815)

a9daf35

Closes pandas-devgh-15230.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Add Index.to_frame method #17815

ENH: Add Index.to_frame method #17815

gfyoung commented Oct 8, 2017

codecov bot commented Oct 8, 2017

codecov bot commented Oct 8, 2017 •

edited

Loading

jschendel Oct 8, 2017

gfyoung Oct 8, 2017

jschendel Oct 8, 2017

gfyoung Oct 8, 2017

jschendel Oct 8, 2017

gfyoung Oct 8, 2017

jschendel Oct 8, 2017

gfyoung Oct 8, 2017

jschendel Oct 8, 2017

gfyoung Oct 8, 2017

jreback Oct 8, 2017

gfyoung Oct 8, 2017

jreback Oct 8, 2017

gfyoung Oct 8, 2017 •

edited

Loading

jreback Oct 9, 2017

jorisvandenbossche Oct 9, 2017

gfyoung commented Oct 8, 2017

jschendel Oct 9, 2017

gfyoung Oct 9, 2017

jschendel commented Oct 9, 2017

jorisvandenbossche left a comment

jorisvandenbossche Oct 9, 2017

jorisvandenbossche Oct 9, 2017

gfyoung Oct 9, 2017

jorisvandenbossche Oct 9, 2017

jorisvandenbossche Oct 9, 2017

gfyoung Oct 9, 2017

jorisvandenbossche Oct 9, 2017

gfyoung Oct 9, 2017

jreback commented Oct 9, 2017

ENH: Add Index.to_frame method #17815

ENH: Add Index.to_frame method #17815

Conversation

gfyoung commented Oct 8, 2017

codecov bot commented Oct 8, 2017

Codecov Report

codecov bot commented Oct 8, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gfyoung Oct 8, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gfyoung commented Oct 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jschendel commented Oct 9, 2017

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Oct 9, 2017

codecov bot commented Oct 8, 2017 •

edited

Loading

gfyoung Oct 8, 2017 •

edited

Loading