+- Python Forum (https://python-forum.io)
+-- Forum: General (https://python-forum.io/forum-1.html)
+--- Forum: News and Discussions (https://python-forum.io/forum-31.html)
+--- Thread: printing interleaved lines from many processes (/thread-18505.html)
printing interleaved lines from many processes - Skaperen - May-21-2019
a new program (no code, yet) will launch many child processes that all will be printing long lines of info to stdout that is passed along from the parent (not piped to the parent). it is essential that each line stay whole and that the interleaving between lines never breaks a line in the middle no matter how stdout gets redirected by the shell when program gets run. lines will not be longer than 1000 characters but they could have Unicode characters in UTF-8 form some day in the future. this will only ever be run on Linux, BSD, Mac, or Unix, never on Windows, DOS, or OS/2. what is the best way, in Python, to ensure that this works?
edit:
to be clear, the parent is passing along stdout that it originally got when it began.
RE: printing interleaved lines from many processes - heiner55 - May-21-2019
Either flush stdout or open stdout without buffering.
RE: printing interleaved lines from many processes - wearsafe - Feb-20-2024
To ensure that the long lines of info printed by child processes are not broken in the middle when redirected by the shell, you can utilize the subprocess module in Python. Specifically, you can use subprocess.Popen to launch the child processes and handle their output.
Here's a basic approach:
Use subprocess.Popen to launch the child processes.
Redirect the stdout of each child process to a pipe.
Read from the pipe in the parent process and print the output.
Here's a simplified example:
python
Copy code
import subprocess
# Launch child process with stdout redirected to a pipe
child_process = subprocess.Popen(['your_child_process_command'], stdout=subprocess.PIPE)
# Read output from the child process line by line
for line in child_process.stdout:
# Print the line without breaking in the middle
print(line.decode('utf-8').rstrip()) # Decode from bytes to string and remove trailing newline
This approach ensures that each line of output from the child process is printed in its entirety, without being broken in the middle, even when redirected by the shell. It also handles Unicode characters encoded in UTF-8 format properly.
You can extend this basic approach to handle multiple child processes by launching them sequentially or in parallel, depending on your requirements. Additionally, you may need to handle error handling and other considerations depending on the specific behavior of your child processes and the environment they are running in.