[Numpy] Load date/time from .txt to 'datetime64' type.

[Numpy] Load date/time from .txt to 'datetime64' type. - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Data Science (https://python-forum.io/forum-44.html)
+--- Thread: [Numpy] Load date/time from .txt to 'datetime64' type. (/thread-41686.html)

[Numpy] Load date/time from .txt to 'datetime64' type. - water - Mar-01-2024

I have date/time data in .txt file likes below format:

2024-2-27, 0:0:0
2024-2-27, 3:7:2
2024-2-27, 4:11:3
2024-2-27, 6:3:5
2024-2-27, 13:10:6
2024-2-27, 20:20:7
2024-2-27, 21:30:9
2024-2-27, 22:40:10
2024-2-27, 23:50:12

I want to load that use numpy.loadtxt() and convert to 'datetime64' type, but 'datetime64' type require string format likes below:

numpy.array(['2020-02-27T15:23:07'], dtype = 'datetime64[s]')

so I must fill some '0', and concatenate date and time use 'T'?

# fill '0'
2024-02-27, 00:00:00
2024-02-27, 03:07:02
2024-02-27, 04:11:03
2024-02-27, 06:03:05
2024-02-27, 13:10:06
2024-02-27, 20:20:07
2024-02-27, 21:30:09
2024-02-27, 22:40:10
2024-02-27, 23:50:12

# concatenate use 'T'
2024-02-27T00:00:00
2024-02-27T03:07:02
2024-02-27T04:11:03
2024-02-27T06:03:05
2024-02-27T13:10:06
2024-02-27T20:20:07
2024-02-27T21:30:09
2024-02-27T22:40:10
2024-02-27T23:50:12

Is it have simply way to do that?

RE: [Numpy] Load date/time from .txt to 'datetime64' type. - Gribouillis - Mar-01-2024

This may not be the fastest method but it seems to work

import datetime as dt
import io
import numpy as np

file = io.StringIO(
    """\
2024-2-27, 0:0:0
2024-2-27, 3:7:2
2024-2-27, 4:11:3
2024-2-27, 6:3:5
2024-2-27, 13:10:6
2024-2-27, 20:20:7
2024-2-27, 21:30:9
2024-2-27, 22:40:10
2024-2-27, 23:50:12
"""
)


def gen(file):
    for line in file:
        t = dt.datetime.strptime(line, "%Y-%m-%d, %H:%M:%S\n")
        yield t.isoformat()


a = np.loadtxt(gen(file), dtype="datetime64[s]")
print(a)

Output:['2024-02-27T00:00:00' '2024-02-27T03:07:02' '2024-02-27T04:11:03'
 '2024-02-27T06:03:05' '2024-02-27T13:10:06' '2024-02-27T20:20:07'
 '2024-02-27T21:30:09' '2024-02-27T22:40:10' '2024-02-27T23:50:12']

RE: [Numpy] Load date/time from .txt to 'datetime64' type. - Danishhafeez - Mar-01-2024

(Mar-01-2024, 08:54 AM)water Wrote: I have date/time data in .txt file likes below format:
2024-2-27, 0:0:0
2024-2-27, 3:7:2
2024-2-27, 4:11:3
2024-2-27, 6:3:5
2024-2-27, 13:10:6
2024-2-27, 20:20:7
2024-2-27, 21:30:9
2024-2-27, 22:40:10
2024-2-27, 23:50:12
I want to load that use numpy.loadtxt() and convert to 'datetime64' type, but 'datetime64' type require string format likes below:
numpy.array(['2020-02-27T15:23:07'], dtype = 'datetime64[s]')
so I must fill some '0', and concatenate date and time use 'T'?
# fill '0'
2024-02-27, 00:00:00
2024-02-27, 03:07:02
2024-02-27, 04:11:03
2024-02-27, 06:03:05
2024-02-27, 13:10:06
2024-02-27, 20:20:07
2024-02-27, 21:30:09
2024-02-27, 22:40:10
2024-02-27, 23:50:12

# concatenate use 'T'
2024-02-27T00:00:00
2024-02-27T03:07:02
2024-02-27T04:11:03
2024-02-27T06:03:05
2024-02-27T13:10:06
2024-02-27T20:20:07
2024-02-27T21:30:09
2024-02-27T22:40:10
2024-02-27T23:50:12
Is it have simply way to do that?

You can achieve this by using the numpy.genfromtxt() function with a custom converter to parse your date/time data and convert it to datetime64 type.

import numpy as np

# Custom converter function to parse date/time strings and convert to datetime64
def datetime_converter(date_str, time_str):
datetime_str = f"{date_str.strip()}, {time_str.strip()}"
return np.datetime64(datetime_str)

# Load data from the text file using genfromtxt with the custom converter
data = np.genfromtxt('your_file.txt', delimiter=',', dtype='U', converters={0: lambda x: x, 1: lambda x: x}, unpack=True)

# Extract date and time columns
dates = data[0]
times = data[1]

# Convert to datetime64 using the custom converter
datetime_array = np.array([datetime_converter(date, time) for date, time in zip(dates, times)])

print(datetime_array)

i hope This code will load your data from the text file, parse the date and time strings, and convert them to datetime64 type using the custom converter function. The resulting datetime_array will contain the datetime values in the desired format.

Best regard
Danish hafeez | QA Assistant

RE: [Numpy] Load date/time from .txt to 'datetime64' type. - water - Mar-01-2024

(Mar-01-2024, 09:30 AM)Gribouillis Wrote: def gen(file):
for line in file:
t = dt.datetime.strptime(line, "%Y-%m-%d, %H:%M:%S\n")
yield t.isoformat()

That seems a good method for this case, thanks.

RE: [Numpy] Load date/time from .txt to 'datetime64' type. - Gribouillis - Mar-01-2024

(Mar-01-2024, 07:25 PM)water Wrote: That seems a good method for this case, thanks.

You can skip a double conversion by creating the array with numpy.fromiter()

import datetime as dt
import io
import numpy as np

file = io.StringIO(
    """\
2024-2-27, 0:0:0
2024-2-27, 3:7:2
2024-2-27, 4:11:3
2024-2-27, 6:3:5
2024-2-27, 13:10:6
2024-2-27, 20:20:7
2024-2-27, 21:30:9
2024-2-27, 22:40:10
2024-2-27, 23:50:12
"""
)


def parse(line):
    return dt.datetime.strptime(line, "%Y-%m-%d, %H:%M:%S\n")


a = np.fromiter((parse(line) for line in file), dtype="datetime64[s]")

print(a)