Processing bulk Jenkins events in parallel
You can enable bulk processing for Jenkins pipeline events from the DevOps Tool integration table and see the events status updates while processing, in the Inbound events table.
Process Jenkins events using bulk flows
New base system scheduled jobs are added to the instance when you upgrade to version 1.26. If you enable bulk processing for your Jenkins events, the following scheduled jobs and corresponding flows are triggered. Currently, each event triggers an orchestration flow which runs the entire duration, one event at a time. Enabling the bulk processing feature initiates the scheduled jobs and starts parallel processing of events in batches based on their state.
In the bulk flow, the earlier orchestration flow is now divided into three stages: Pre-processing, Processing and Post-processing. Events are grouped by the stage they are in and processed in parallel instead of one event at a time.
- Bulk Processing-Pre
- Inbound events for Jenkins in New/Retry state are moved to Ready state if the tool ID and connection status are valid. Verifies that no flow executions exist in Flow Designer.
- Bulk Event Processor
- Bulk processing phase picks all the events from Ready state and transforms the payload and marks them as Transformed on completion.
- Bulk Processing-Post
- This job picks up all the events from Transformed state and moves them to Processed state. If event should wait, then it should update them as Transformed state and update the Waiting Reason field.
- Bulk Processing-Monitor
- Resets any workflow if event is stuck during bulk_processing. When the Bulk processing is set to True from False state, new events are assigned the new bulk flow. If Bulk processing is set to False from True the state of the inbound event is set to New and the old workflow is assigned.