spack/lib/spack/llnl/util
Tom Scogland 4a7b0afde2
Log performance improvement (#23925)
* util.tty.log: read up to 100 lines if ready

Rework to read up to 100 lines from the captured stdin as long as data
is ready to be read immediately.  Adds a helper function to poll with
`select` for ready data.  This showed a roughly 5-10x perf improvement
for high-rate writes through the logger with relatively short lines.

* util.tty.log: Defer flushes to end of ready reads

Rather than flush per line, flush per set of reads.  Since this is a
non-blocking loop, the total perceived wait is short.

* util.tty.log: only scan each line once, usually

Rather than always find all control characters then substitute them all,
use `subn` to count the number of control characters replaced.  Only if
control characters exist find out what they are.  This could be made
truly single pass with sub with a function, but it's a more intrusive
change and this got 99%ish of the performance improvement (roughly
another 2x in some cases).

* util.tty.log: remove check for `readable`

Python < 3 does not support a readable check on streams, should not be
necessary here since we control the only use and it's explicitly a
stream to be read.
2021-05-31 20:33:14 -07:00
..
tty Log performance improvement (#23925) 2021-05-31 20:33:14 -07:00
__init__.py copyrights: update all files with license headers for 2021 2021-01-02 12:12:00 -08:00
argparsewriter.py copyrights: update all files with license headers for 2021 2021-01-02 12:12:00 -08:00
filesystem.py Python 3.10 support: collections.abc (#20441) 2021-02-01 11:30:25 -06:00
lang.py Put a module object in sys.modules before executing module code (#23269) 2021-05-06 11:53:40 +02:00
link_tree.py copyrights: update all files with license headers for 2021 2021-01-02 12:12:00 -08:00
lock.py Use gethostname() instead of getfqdn() for lock debug mode 2021-04-15 00:01:41 -07:00
multiproc.py copyrights: update all files with license headers for 2021 2021-01-02 12:12:00 -08:00