+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: multi-threading error in minimal script (/thread-20265.html)
multi-threading error in minimal script - Skaperen - Aug-02-2019
i am getting the error shown below in a minimal little script that runs just 2 threads:
Output:lt2a/forums /home/forums 1> cat -n trytwocommands.py
1 #!/usr/bin/env python3
2 import subprocess,threading
3 one = threading.Thread(target=subprocess.call,args=['sleep','10001'])
4 two = threading.Thread(target=subprocess.call,args=['sleep','10002'])
5 one.start()
6 two.start()
7 print('started 2 threads, now joining them',flush=1)
8 one.join()
9 two.join()
10 print('joined 2 threads, now done',flush=1)
lt2a/forums /home/forums 2> py3 trytwocommands.py
started 2 threads, now joining them
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.6/subprocess.py", line 287, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.6/subprocess.py", line 629, in __init__
raise TypeError("bufsize must be an integer")
TypeError: bufsize must be an integer
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.6/subprocess.py", line 287, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.6/subprocess.py", line 629, in __init__
raise TypeError("bufsize must be an integer")
TypeError: bufsize must be an integer
joined 2 threads, now done
lt2a/forums /home/forums 3>what is bufsize and what set it to a non-integer? is there a way i can set it? or is subprocess.call() not thread-safe?
RE: multi-threading error in minimal script - Gribouillis - Aug-03-2019
It seems to me that you should pass
,args=[['sleep','10001']] otherwise python "thinks" that the args to Popen is 'sleep' and that '10001' is the bufsize argument of Popen.
RE: multi-threading error in minimal script - Skaperen - Aug-03-2019
i was making sure subprocess.call (via Popen) can run commands in separate processes from subthreads so i can, at least, consider threads as a way to avoid having 2 processes per concurrent command. it still won't let me leave the last bunch of commands running, but for now this gets this project going (faster backups to AWS S3).
#!/usr/bin/env python3
import subprocess,threading
m = 64
threads = [threading.Thread(target=subprocess.call,args=(['sleep',str(10000+x)],)) for x in range(1,m+1)]
[threads[x].start() for x in range(m)]
print(f'started {m} threads',flush=1)
[threads[x].join() for x in range(m)]
print(f'joined {m} threads',flush=1)