Python: How to execute shell commands / scripts efficient

Imagine the following scenario: every-night we execute a task, the task is a shell script that copies some files remotely.

Depending many factors, the task might not complete successfully. So, there are many mornings that we go to the office, we check the task results and often we need to re-run the task. this is time consuming and inefficient.

How we can do our lives easier? how we can automate the re-run of the task? I wrote a small Python script that executes a task, which might be a shell command or a script, if the task returns an exit code that is not equal to “0” (which usually indicates execution success), the wrapper script re-runs the task. The task will stop executing in case of success or in case that failures exceed an optional limit.

Also there is a “wait” option between execution tries for the case that a delay between executions is needed.

Examples:

• -c: “The command or script to be executed”

c:\Users\kpatr\OneDrive\Desktop\scripts\guardian>python guardian.py -c "dir"
2020-06-18 13:55:25.460505 - SUCCESS - STDIN: dir
STDOUT:
  Volume in drive C has no label.
 Volume Serial Number is 36FC-5968

 Directory of c:\Users\kpatr\OneDrive\Desktop\scripts\guardian

18/06/2020  13:55    <DIR>          .
18/06/2020  13:55    <DIR>          ..
18/06/2020  13:55             1,600 guardian.py
               1 File(s)          1,600 bytes
               2 Dir(s)  357,928,259,584 bytes free

• -w: wait time between execution failures in seconds. default: 30s

• -r: maximum number of execution tries in case of failure. default: 1 repeat

c:\Users\kpatr\OneDrive\Desktop\scripts\guardian>python guardian.py -c "dir -a" -w 3 -r 3
2020-06-18 13:56:36.600020 - TRY - 1: FAILURE: dir -a STDERR: File Not Found

2020-06-18 13:56:39.629230 - TRY - 2: FAILURE: dir -a STDERR: File Not Found

2020-06-18 13:56:42.652491 - TRY - 3: FAILURE: dir -a STDERR: File Not Found

2020-06-18 13:56:42.652491 - GIVING UP: Maximum repeations reached for dir -a

How to repeat until task execution success:

Passing a negative value to the -r parameters will do the trick, the task will continue executing until success.

c:\Users\kpatr\OneDrive\Desktop\scripts\guardian>python guardian.py -c "dir -a" -w 3 -r -1
2020-06-18 14:07:27.260575 - TRY - 1: FAILURE: dir -a STDERR: File Not Found

2020-06-18 14:07:30.282983 - TRY - 2: FAILURE: dir -a STDERR: File Not Found

2020-06-18 14:07:33.303639 - TRY - 3: FAILURE: dir -a STDERR: File Not Found

Source: Medium.com