- 03 Nov, 2016 1 commit
-
-
Jason Rhinelander authored
The increment was only happening when sleeping was required, but obviously needs to happen always.
-
- 02 Nov, 2016 2 commits
-
-
Jason Rhinelander authored
The previous code was obtaining the jobno and opening the log before any necessary sleeping, which two undesirable effects: 1. The "Starting job n" messages looked fairly random when jobs were delayed. 2. The initial batch of jobs were allocated not as hosts are actually available, but when hosts are available after a delay. For a fast job with some hosts having a large number of CPUs, this means the job could take unnecessarily long: e.g. imagine a 'hostA=1 hostB=99' with an instantaneous job (and connection) executing 100 jobs: all jobnos would be instantly grabbed by the 99 threads, but the last job on hostB wouldn't run for 99 DELAY periods, even though the optimal solution is 50 jobs each (and a total time of 50 DELAYs instead of 99 DELAYs). This fixes the problem by checking and obtaining a jobno after finishing the delay.
-
Jason Rhinelander authored
abort_jobs was only checked when entering the job loop, but if a job entered the loop then had to wait, it didn't check abort_jobs again, so the delayed jobs would start up even after a Ctrl-C was received.
-
- 25 Oct, 2016 1 commit
-
-
Jason Rhinelander authored
If a host supports many simultaneous jobs (e.g. a 20-core system), establishing all 20 connections simultaneously is a bad idea; this adds a minimum delay (defaulting to 0.25) between connections to the same host.
-
- 21 May, 2016 1 commit
-
-
Jason Rhinelander authored
FIRST lets you control the first PARALLEL_JOBNO (instead of always starting from 1). This is useful when you do something like '$((PARALLEL_JOBNO % 10))' to select one of 10 different cases in the job, so that you can restart exactly where you left off, instead of always having to restart at case '1'. Added --help argument (also HELP, -h, -?) to display full help. Bad/no arguments now displays a short usage message without the help details. Allowed HOSTS, PER_HOST, and FIRST to be set through environment variables, prefixed with PARALLEL_, e.g. $ export PARALLEL_FIRST=27 $ parallel-runner 10 'echo hi' now does the same thing as: $ parallel-runner FIRST=27 10 'echo hi' Command line arguments have highest precedence, then env vars, then file vars.
-
- 20 May, 2016 1 commit
-
-
Jason Rhinelander authored
-
- 19 May, 2016 1 commit
-
-
Jason Rhinelander authored
- Add finish date and total time to output - Add overall job speed - Moved hosts stats after "Finished" and overall summary - Ident hosts for better appearance - Fixed bad grammar in "Finished: 1 successful jobs, 1 failures." - Fixed too many blank lines in output
-
- 18 May, 2016 6 commits
-
-
Jason Rhinelander authored
My previous attempts weren't working: stdout was still occasionally lost. This redoes it to not require pseudoterminal allocation at all by instead using a pipe to send input to the remote process, and using an extra subprocess that blocks for input (which never comes), and kills the subprocess if it complete (i.e. because of an EOF). It seems to work: I haven't lost any output yet. This also adds some various logging improvements, showing connections/disconnections in the log as well as exit status (plus a reinterpretation of bash's 128+n as a N terminal signal).
-
Jason Rhinelander authored
The previous commit didn't fix things--some stdout could still be lost. This is another attempt to fix the problem by making the log file unbuffered.
-
Jason Rhinelander authored
Otherwise there is potential conflict since the parent and child both have the file open--but this can cause data loss, since they'll not have the same file position after the fork. So reopen for each job, and close (in the parent) immediately after the fork.
-
Jason Rhinelander authored
-
Jason Rhinelander authored
-
Jason Rhinelander authored
(Moved from one of my private repositories).
-