Skip to content

WIP: Make .item() and .astype(object) return NumPy scalars for datetime and timedelta#18683

Closed
h-vetinari wants to merge 4 commits into
numpy:mainfrom
h-vetinari:consistent_datetime64
Closed

WIP: Make .item() and .astype(object) return NumPy scalars for datetime and timedelta#18683
h-vetinari wants to merge 4 commits into
numpy:mainfrom
h-vetinari:consistent_datetime64

Conversation

@h-vetinari

Copy link
Copy Markdown
Contributor

This is to avoid implicitly casting e.g. datetime64 to (python-)datetimes when
using operations - like .astype(object) - that are expected not to change the
type. Fixes #12550.

Co-Authored-By: Sebastian Berg [email protected]


The diff was provided by @seberg, but is likely incomplete (see link). Opening this PR to hopefully start iterating towards completion. Help welcome - I'll happily give push access to my fork for those who want to contribute.

CC @jbrockmendel

@eric-wieser eric-wieser changed the title Make .item() and .astype(object) return NumPy scalars Make .item() and .astype(object) return NumPy scalars for datetime and timedelta Mar 28, 2021
@eric-wieser

Copy link
Copy Markdown
Member

Doesn't this change remove the only way to convert from a numpy date scalar to a built-in one?

@h-vetinari h-vetinari changed the title Make .item() and .astype(object) return NumPy scalars for datetime and timedelta WIP: Make .item() and .astype(object) return NumPy scalars for datetime and timedelta Mar 28, 2021
@h-vetinari h-vetinari marked this pull request as draft March 28, 2021 11:19
@h-vetinari

Copy link
Copy Markdown
Contributor Author

Doesn't this change remove the only way to convert from a numpy date scalar to a built-in one?

If it does, then we'll have to add a method. I forgot to mark this PR as a draft (actually, even less: just a start for iterating on one based on @seberg's input) - I hope that's clearer now.

@seberg

seberg commented Mar 28, 2021

Copy link
Copy Markdown
Member

Yeah, you are right. Need to change only the "to object" casting... to not rely on getitem. I have a half plan to make the getpyobject and getitem two distinct methods more clearly (since we have both right now). But the thing is that most of our scalars do use getitem for their object casts (i.e. float64arr.astype(object) returns python floats!

So, this needs special cased casting, so you need to change the casting in this file, and then override that the casting is actually always used as well!

@jbrockmendel

Copy link
Copy Markdown
Contributor

Doesn't this change remove the only way to convert from a numpy date scalar to a built-in one?

I'm not sure if this is a pattern we want to encourage, but this works in 1.20.2

import numpy as np
from datetime import datetime

arr = np.arange(10).view("M8[s]")

>>> arr[2].astype(datetime)
datetime.datetime(1970, 1, 1, 0, 0, 2)

@seberg

seberg commented Mar 29, 2021

Copy link
Copy Markdown
Member

The point is that arr.item() (and arr.tolist() I guess) should probably return the Python scalar (or error), while arr.astype(object) does currently the same as arr.item(), but based on the other issue the idea was that it should change to return NumPy date-time scalars.
The patch would have changed both, but I guess the idea was to only change the astype().

Note that .astype(datetime) is the same as .astype(object).

This is to avoid implicitly casting e.g. datetime64 to (python-)datetimes when
using operations - like .astype(object) - that are expected not to change the
type. Fixes numpy#12550.

Co-Authored-By: H. Vetinari <[email protected]>
@h-vetinari h-vetinari force-pushed the consistent_datetime64 branch from e53b1e5 to d8d1af5 Compare February 5, 2022 06:44
@h-vetinari h-vetinari force-pushed the consistent_datetime64 branch from d8d1af5 to 2a56b7c Compare February 5, 2022 06:55
@seberg

seberg commented Feb 21, 2023

Copy link
Copy Markdown
Member

I suspect this is be de-facto superseded by @jbrockmendel new start in gh-23205 (and otherwise is stalled anyway). So closing for now, we can reopen at any point though!

@seberg seberg closed this Feb 21, 2023
@h-vetinari

Copy link
Copy Markdown
Contributor Author

Yeah, sorry, this never made it to the top of my to-do list 🥲

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: astype(object) downcasts for datetime-dtype

4 participants