run a command, capture output, what form do you want?

[Skaperen]

when you run a command with the intent to get the output of that command, in what form would be most convenient for you?

1. what if the command is a network ping and you want each ping result as soon as the command outputs it? i already know i would not want to wait for the command to end and get all the output after that. well, in most cases, anyway.

2. what if the command is a text interactive command?

3. what if the command outputs one character at a time?

4. what if the output is binary with any byte code (0..255)?

would you want only whole lines, one line at a time?

would you want the whole output as one giant string?

do you ever need to send input to the command?

do you ever need to send it a Ctrl-C?

[Axel_Erfurt]

If I want it easy and with little effort I use Gtk and embed VTE Terminal.

[Gribouillis]

what if the command is a network ping and you want each ping result as soon as the command outputs it?

Output:
λ ping python-forum.io | tee foo.txt
PING python-forum.io(2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3)) 56 data bytes
64 bytes from 2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3): icmp_seq=1 ttl=59 time=11.8 ms
64 bytes from 2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3): icmp_seq=2 ttl=59 time=11.4 ms
64 bytes from 2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3): icmp_seq=3 ttl=59 time=11.3 ms
64 bytes from 2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3): icmp_seq=4 ttl=59 time=11.5 ms
^C
λ cat foo.txt
PING python-forum.io(2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3)) 56 data bytes
64 bytes from 2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3): icmp_seq=1 ttl=59 time=11.8 ms
64 bytes from 2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3): icmp_seq=2 ttl=59 time=11.4 ms
64 bytes from 2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3): icmp_seq=3 ttl=59 time=11.3 ms
64 bytes from 2606:4700:3037::ac43:a8e3 (2606:4700:3037::ac43:a8e3): icmp_seq=4 ttl=59 time=11.5 ms

[Skaperen]

now try calling something in some Python3 code and capturing each ping response and output the time since the previous response (beginning of command for first response). the unit of delta time can be in milliseconds. at the end give the delta stats: lowest, median, average, highest. the user running this command should not need to know that it calls the "ping" command. the question is, what way would you like to have your code get this data?

[ndc85430]

Why would you run "ping" and capture the output instead of using, say icmplib? You'd have the data in an object that you could access then.

[Gribouillis]

the question is, what way would you like to have your code get this data?

The way you describe the problem, it it clear that the subprocess' stdout becomes a source of data that must be read as soon as it is available, so the following should work

1. Create an instance of selectors.DefaultSelector and register EVENT_READ for the subprocess' stdout in this instance.
   
2. Call the select() method, which returns a real time sequence of when data has been produced by the subprocess. For each event in the sequence, invoke a callback that reads in the subprocess stdout and attach a time information to the data by calling for example time.perf_counter().
   
3. If the program needs to do something else at the same time, create a thread devoted to reading the subprocess' output. Clearly the thread that receives the data must not do anything else, otherwise the time information will be damaged.
   

[snippsat]

now try calling something in some Python3 code and capturing each ping response and output the time since the previous response (beginning of command for first response). the unit of delta time can be in milliseconds. at the end give the delta stats: lowest, median, average, highest

icmplib(The best Python implementation ICMP protocol) will give timings of diffrent ping times used.

from icmplib import ping

host = ping('python-forum.io', count=4)
print(host)

Output:
  104.21.27.41
------------------------------------------------------------
  Packets sent:     4
  Packets received: 4
  Packet loss:      0.0%
  Round-trip times: 24.0 ms / 29.25 ms / 33.0 ms
  Jitter:           5.0 ms
----------------------------------------------------------

# min, avg, max, jitter(variance of the latency of packets flowing through the network)
>>> host.min_rtt
29.0
>>> host.avg_rtt  
32.0
>>> host.max_rtt
35.0
>>> host.jitter
5.333

[Skaperen]

can you print something about the timing of each ping response immediately after the response is received?

the intent of the original question is getting lost. it's about how you get the output lines from a command being run, when you need to process each separate output just after it is received from that command. what would the preferred way be?

edit:

ping is not a good example if it has a function to provide this. it's about implementing something not yet implemented that can be done using the output of some command with processing done immediately after each line of output.