Python Forum
read_csv error and rows/columns missing
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
read_csv error and rows/columns missing
#1
I thought I got everything right till I ended up with a different file to read. Damn for 2 days I'm struggling with it. Help, please.
link to the data:file.zip

Thks for your time

# Location of all files 
file_folder = 'path_on_your_computer'

# Save the files into a list (when more than 2)
list_raw_files = [f for f in listdir(file_folder) if isfile(join(file_folder, f))]

# Load the right/given file
for raw_file in list_raw_files:   
    
    #  Check the file 
    if raw_file.startswith('130'): #print (raw_file)
        
        temp_list = []
        
        for chunk in pd.read_csv(file_folder + raw_file, sep = ';', header = None, chunksize = 20000, error_bad_lines = False , low_memory=False):
            
            temp_list.append(chunk)
        
        data = pd.concat(temp_list, axis = 0)
        
        del temp_list

data.head(30)
I have this Error/Warning:
IOPub data rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
--NotebookApp.iopub_data_rate_limit.

Current values:
NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec)
NotebookApp.rate_limit_window=3.0 (secs)

and the data is not complete!
link to my result: output
Reply


Messages In This Thread
read_csv error and rows/columns missing - by karlito - Nov-05-2019, 10:04 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Merging rows and adding columns based on matching index pythonnewbie78 3 907 Dec-24-2023, 11:51 AM
Last Post: Pedroski55
  Pandas read_csv markf7319 0 1,309 Mar-03-2022, 04:59 AM
Last Post: markf7319
  groupby on var with missing values error zenvega 0 1,826 May-07-2021, 07:40 PM
Last Post: zenvega
  pandas read_csv can't handle missing data mrdominikku 0 2,549 Jul-09-2020, 12:26 PM
Last Post: mrdominikku
  Checking a filename before reading it with pd.read_csv karlito 2 2,266 Oct-30-2019, 09:46 AM
Last Post: karlito
  Drop rows if a set of columns has a value dervast 1 2,035 Sep-12-2019, 04:18 PM
Last Post: sd_0912
  display graph in columns and rows william888 1 1,903 Jul-02-2019, 10:19 AM
Last Post: dataman
  utf-8 error with pandas read_csv logues 0 3,916 Oct-23-2018, 05:25 PM
Last Post: logues
  Dropping all rows of multiple columns after the max of one cell Thunberd 2 3,019 Jun-01-2018, 10:18 PM
Last Post: Thunberd
  pandas read_csv, numbers in footer mechanic310 1 2,862 May-22-2018, 10:38 AM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020