Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
chopping a bytearray
#1
i have a large bytearray that i am writing to disk. i expect in most cases it will write the entire array in one call to binaryoutputfile.write(mybytearray). but, sometimes, there can be system conditions (it might be on a networked file system) that cause an incomplete write. simple code to do that could be:
# write the entire buffer
        while outbuf:
            rem=outfile.write(outbuf)
            if not rem:
                break
            outbuf[:-rem]=bytearray()
this code causes byte values in outbuf to be shifted by the slice assignment. it could be made to operate faster by calculating more numbers to slice out just what remains and not do a slice assignment. i don't have code for that, yet. but i am wondering if more complicated code to improve performance in rare cases could be considered reasonably pythonic.

note, i am not knowledgeable about how CPython has implemented slice assignments. i'm just imagining ways to do it in C.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
This code looks a little strange to me because usually write() functions return the amount of data written (see for example io.BufferedWriter.write()). This data written should be at the beginning of outbuf and the code seems to chop the end of outbuf. Also the variable name 'rem' is misleading, it looks like 'remaining data' which is wrong.

Apart from that, you could perhaps write a separate function
def writeall(outfile, outbuf):
    ...

that would write all of outbuf into outfile, similarly to socket.sendall() vs socket.send(). Of course this function would read computed slices of outbuf without ever reassigning outbuf or one of its slices.
Reply
#3
my aged memory must be corrupt and mixed that up with the system call. i like the distinct function idea. i will do that. i could even add that as a method for my zopen and ztopen classes.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  bytearray (and bytes) in Python2 Skaperen 3 2,832 Jun-06-2019, 05:43 AM
Last Post: heiner55
  os.path.join, bytearray, python2 vs python3 Skaperen 0 1,987 Jul-06-2018, 01:19 AM
Last Post: Skaperen

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020