Builder rsyslog-solaris10-i386 Build #1896

Results:

Failed

SourceStamp:

BuildSlave:

Reason:

The SingleBranchScheduler scheduler named 'schedule-rsyslog-solaris10-i386' triggered this build

Steps and Logfiles:

1. git updating ( 1 mins, 2 secs )
   1. stdio
2. shell  
   1. - no logs -
3. shell_1  
   1. - no logs -
4. shell_2  
   1. - no logs -
5. shell_3  
   1. - no logs -

Build Properties:

Forced Build Properties:

Responsible Users:

1. Rainer Gerhards

   rgerhardsohnoyoudont@adiscon.com

Timing:

All Changes:

:

1. Change #223466

   Category	rsyslog
   Changed by	Rainer Gerhards <rgerhardsohnoyoudont@adiscon.com>
   Changed at	Sat 01 Mar 2025 14:01:20
   Repository	https://github.com/rsyslog/rsyslog.git
   Project	rsyslog
   Branch	master
   Revision	ad1fd213a79bf802a2af2258c380378d4d69fdb2

   Comments

   imtcp: major multithreading and performance improvements
   This commit significantly enhances imtcp by introducing a fully
   functional worker thread pool, enabling true multi-threaded
   processing for better scalability under high loads. This is
   particularly beneficial when using TLS connections.
   
   Notable changes:
   - Implemented a complete worker pool for imtcp.
   - Introduced the `workerthreads` config parameter for tuning
     concurrency.
   - Improved epoll efficiency by enabling edge-triggered mode.
   - Added starvation handling via `starvationProtection.maxReads`.
   - Refactored session accept logic and optimized network object
     handling.
   - Removed an obsolete network driver layer for event notification.
   - Fixed multiple issues related to message timing, EPOLLERR
     handling, and tests.
   - Improved performance in poll() mode by reducing redundant
     allocations.
   - Introduced new CI tests for imtcp without epoll.
   - Allowed disabling imtcp tests via a new configure switch.
   - Added new impstats counters for worker thread pool statistics.
   
   Details:
   - The worker pool replaces an outdated experimental
     implementation.
   - If `workerthreads=1`, no worker pool is created to minimize
     context switches.
   - Moves worker pool variables inside `tcpsrv` instance to
     prevent conflicts.
   - Extracts session `accept()` logic into a dedicated function
     for clarity.
   - Fixes message ordering inconsistencies in multi-threaded
     scenarios.
   - Properly handles `EPOLLERR` notifications to improve error
     resilience.
   - Optimizes poll() mode by avoiding unnecessary reallocation
     of file descriptors.
   - Replaces the old network driver layer for event notification
     with a streamlined solution.
     - Now uses **conditional compilation** to select the best
       method (epoll or poll) at build time.
     - This significantly reduces code complexity, improves
       maintainability, and boosts performance.
   - The previous "thread pool" was a rough experiment that did
     not perform significantly better than single-threaded mode.
     - The **new implementation** allows multiple worker threads
       on platforms with `epoll`.
     - On non-epoll systems, an optimized **poll() based
       single-threaded approach** is used, which is expected to
       perform better than the old "thread pool."
   - Adds `pthread_setname_np` only when available to improve
     portability.
   - Fixes test cases that assumed strict message timing, which
     was unreliable.
   - Reduces test parallelism for TSAN CI runs to prevent
     resource exhaustion.
   - Moves a test case to `imdiag` to ensure stable execution.
   - Provides a new CI environment to verify `imtcp` behavior
     without epoll.
   - Introduces `--enable-imtcp-tests` configure switch for test
     flexibility.
   - Improves debug logging and adds better error handling for
     worker pool startup.
   
   New configuration parameters:
   - `workerthreads`: Defines the number of worker threads for
     imtcp. If set to 1, no worker pool is created.
   - `starvationProtection.maxReads`: Defines the maximum number
     of consecutive reads a worker can perform before being
     interrupted to allow other sessions to be processed.
   
   New impstats counters (emitted only when `workerthreads > 1`):
   - `runs`: Number of times the worker thread has been invoked.
   - `read`: Number of read calls performed by the worker.
     For TLS, this includes read/write calls.
   - `accept`: Number of `accept()` calls handled by the worker.
   - `starvation_protect`: Number of times a socket was sent
     back to the queue due to reaching the maximum number of
     consecutive requests, ensuring fair scheduling of sessions.
   
   These changes significantly enhance rsyslog’s TCP handling
   performance and stability, particularly in high-volume
   environments.
   
   Closes #5529, #5532, #5578, #5580.

   Changed files

   • .github/workflows/run_checks.yml
   • configure.ac
   • plugins/imdiag/imdiag.c
   • plugins/imtcp/imtcp.c
   • runtime/Makefile.am
   • runtime/netstrms.c
   • runtime/nsd.h
   • runtime/nsd_gtls.c
   • runtime/nsd_ossl.c
   • runtime/nsd_ptcp.c
   • runtime/nsdpoll_ptcp.c
   • runtime/nsdpoll_ptcp.h
   • runtime/nsdsel_gtls.c
   • runtime/nsdsel_gtls.h
   • runtime/nsdsel_ossl.c
   • runtime/nsdsel_ossl.h
   • runtime/nsdsel_ptcp.c
   • runtime/nsdsel_ptcp.h
   • runtime/nspoll.c
   • runtime/nspoll.h
   • runtime/nssel.c
   • runtime/nssel.h
   • runtime/rsyslog.h
   • runtime/tcps_sess.c
   • runtime/tcps_sess.h
   • runtime/tcpsrv.c
   • runtime/tcpsrv.h
   • runtime/typedefs.h
   • runtime/wti.c
   • runtime/wtp.c
   • tests/Makefile.am
   • tests/diag.sh
   • tests/dynfile_invld_sync.sh
   • tests/imtcp-impstats-single-thread.sh
   • tests/imtcp-impstats.sh
   • tests/imtcp-starvation-0.sh
   • tests/imtcp-starvation-1.sh
   • tests/manytcp.sh
   • tests/pmnormalize-basic.sh
   • tests/pmnormalize-rule.sh
   • tests/sndrcv_tls_ossl_anon_rebind.sh
   • tests/tcp-msgreduc-vg.sh

2. Change #223467

   Category	rsyslog
   Changed by	Rainer Gerhards <rgerhardsohnoyoudont@adiscon.com>
   Changed at	Sat 01 Mar 2025 18:23:57
   Repository	https://github.com/rsyslog/rsyslog.git
   Project	rsyslog
   Branch	master
   Revision	5a3ecfea429cb249119243f17b1759e8be0ac62c

   Comments

   Merge pull request #5497 from rgerhards/imtcp-mt-refactor
   tcpsrv: improve multithreading

   Changed files

   • no files