Python stdout not redirected by bash '>'

[RoGeorge]

Wanted to save the console output from terminal to a file, by redirecting stdout with '>', like wither of these lines, and none was working:

Code: [Select]

./service_explorer.py --name SensorTag > conout.txt
python3 ./service_explorer.py --name SensorTag > conout.txt
python3 -u ./service_explorer.py --name SensorTag > conout.txt
- either of the above lines will produce an empty file 'conout.txt', all the text is seen in the terminal only
- added -u (unbuffered) in the shebang or in the command line doesn't fix it, so I don't think it is about flushing buffers
- other redirections from a Python test of a single line 'print ("ok")', does work, the text 'ok' is put in 'conout.txt'

- found something that might be related with the issue, though I'm not sure if this is the cause, or how to overcome it
https://bugs.python.org/issue44158

The troubling Python file, 'service_explorer.py', is an example from a python module for Bluetooth LE, 'bleak', and the code is working OK in a terminal:  https://github.com/hbldh/bleak/blob/master/examples/service_explorer.py

For now, as a workaround I can manually copy/paste the text from the terminal, but I plan to use 'bleak' to write code that will print to stdout in order to be redirected and/or piped to other command line tools.

- 1.  Why bash redirection is not working here?
- 2.  What to do, to produce redirectable stdout (for other Python programs using the 'bleak' module)?

[DiTBho]

try "2>"
(error stream)

[RoGeorge]

Doh, turns out python logger writes to stderr, indeed, that solves it, thank you. 

[Veteran68]

and "2>&1" if you want to combine stderr with stdout to redirect both at the same time

[RoGeorge]

Redirection can get tricky in bash.  That 2>&1 is a particularly nasty one, depending on if it is written before or after the >file redirecting.

The name "redirection" is misleading, because that's the goal, not the operation.  The operation is that 2>&1 makes a copy of the STDOUT file handler (whatever that handler might be at that moment), and assigns it to the STDERR file handler.  bash interprets redirections from left to right, and a file handler may change during the same bash line, so we get behavior like :

Code: [Select]

# /tmp exists and goes to STDOUT, but /xyz does not and goes to STDERR, both visible in the terminal
#
$ ls -ld /tmp /xyz
ls: cannot access '/xyz': No such file or directory
drwxrwxrwt 23 root root 27 Aug 25 21:05 /tmp


# ls -ld /tmp /xyz 2>&1 >aaafile
#
# this might look like STDERR is "redirected" to STDOUT, then STDOUT is "redirected" to aaafile
#    but it is not so.  In fact, the handler of STDOUT is copied as the STDERR handler,
#    then the handler of aaafile is copied as the STDOUT handler,
#    the effect is that errors will appear on the screen, while console output will go into aaafile,
#    so it does not "redirect" both the STDOUT and the STDERR to aaafile, as intended  :o/
#
$ ls -ld /tmp /xyz 2>&1 >aaafile
ls: cannot access '/xyz': No such file or directory
$ cat aaafile
drwxrwxrwt 23 root root 27 Aug 25 21:05 /tmp


# ls -ld /tmp /xyz >aaafile 2>&1
#
# this will "redirect" both console output and console errors to the aaafile, because
#    first the STDOUT is made to point to aaafile
#    then the STDOUT handler (which is now pointing to aaafile) is copied into the STDERR handler
#    so we end up with both the STDOUT and the STDERR pointing to aaafile, as wished
#
$ ls -ld /tmp /xyz >aaafile 2>&1
$ cat aaafile
ls: cannot access '/xyz': No such file or directory
drwxrwxrwt 23 root root 27 Aug 25 21:05 /tmp

 

[golden_labels]

Note that in bash there are also &>, &>> and |&, which redirect both stdout and stderr at the same time:

Code: [Select]

command &> file
command &>> file
command |& command

[RoGeorge]

That one would have worked, too.  Since we are at it:


Bash Redirection Cheat Sheet from https://catonmat.net/bash-redirections-cheat-sheet