Python Forum
launch processes from threads
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
launch processes from threads
#1
i am refactoring a program that dynamically runs many processes at the same time. these processes are other programs (often to download, upload, compress, or uncompress something). a 2nd process runs for each "task" to start those process and wait for them to finish (or restart them if soft errors happen). this means each task is running 2 processes. i am hitting process limits too early. so i am thinking about threads to run the control logic for each task. but this would mean launching processes from threads which i recall reading that is not supported. can someone tell me of the current status of and if a newer version of Python3 can now do this. if not, i am thinking to have an extra process that does all the process work (launching, waiting) based on requests via pipes and sending responses back via a pipe. maybe there would be a pipe per task or the task ID is sent with each message over the pipe. ideas? FYI, i have done similar in C w/o threads.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Messages In This Thread
launch processes from threads - by Skaperen - Feb-09-2024, 12:56 AM
RE: launch processes from threads - by Gribouillis - Feb-09-2024, 04:24 AM
RE: launch processes from threads - by Skaperen - Feb-10-2024, 10:58 PM
RE: launch processes from threads - by DeaD_EyE - Feb-14-2024, 04:00 PM
RE: launch processes from threads - by Skaperen - Feb-17-2024, 12:40 AM
RE: launch processes from threads - by wearsafe - Feb-20-2024, 01:40 AM
RE: launch processes from threads - by Skaperen - Feb-20-2024, 06:35 AM
RE: launch processes from threads - by Skaperen - Feb-20-2024, 06:44 AM
RE: launch processes from threads - by DeaD_EyE - Feb-20-2024, 07:07 AM
RE: launch processes from threads - by Skaperen - Feb-21-2024, 01:16 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  printing interleaved lines from many processes Skaperen 2 2,171 Feb-20-2024, 01:48 AM
Last Post: wearsafe
  order to call Popen for 2 piped processes Skaperen 0 1,207 Oct-22-2020, 11:31 PM
Last Post: Skaperen
  capture stdout from child processes Skaperen 0 3,365 Oct-30-2019, 12:11 AM
Last Post: Skaperen
  [split] launch .PY program sparkz_alot 4 6,047 Sep-28-2016, 05:16 PM
Last Post: sparkz_alot

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020