Jan-06-2018, 04:49 PM
(This post was last modified: Jan-06-2018, 04:49 PM by Gribouillis.)
It looks like a bug in python(!). If you send a complete reader's side example, I can try it here. Obviously the
For a more elaborate diagnosis, you could run the following file at the beginning of the reader's program
.buffer
member in class codecs.BufferedIncrementalDecoder
should never become None
. See the class here.For a more elaborate diagnosis, you could run the following file at the beginning of the reader's program
import codecs from codecs import IncrementalDecoder class BufferedIncrementalDecoder(IncrementalDecoder): """ This subclass of IncrementalDecoder can be used as the baseclass for an incremental decoder if the decoder must be able to handle incomplete byte sequences. """ def __init__(self, errors='strict'): IncrementalDecoder.__init__(self, errors) # undecoded input that is kept between calls to decode() self.buffer = b"" def _buffer_decode(self, input, errors, final): # Overwrite this method in subclasses: It must decode input # and return an (output, length consumed) tuple raise NotImplementedError def decode(self, input, final=False): # decode input (taking the buffer into account) data = self.buffer + input (result, consumed) = self._buffer_decode(data, self.errors, final) # keep undecoded input until the next call self.buffer = data[consumed:] return result def reset(self): IncrementalDecoder.reset(self) self.buffer = b"" def getstate(self): # additional state info is always 0 return (self.buffer, 0) def setstate(self, state): # ignore additional state info self.buffer = state[0] # Replace the class in module codecs by our own class codecs.BufferedIncrementalDecoder = BufferedIncrementalDecoderThen the codecs module will use your own version of the class. You can now tweak the class' code to detect exactly which bytes are read, when does the buffer becomes None, etc. For example you could turn
.buffer
into a property and raise an exception as soon as it receives the value None
.