Skip to content

Tweak delta to make test less flaky on CI. #10429

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
AlexeyKuznetsov-DD wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Tweak delta to make test less flaky on CI. #10429

AlexeyKuznetsov-DD wants to merge 1 commit into from

Conversation

@AlexeyKuznetsov-DD
Copy link
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD commented Jan 22, 2026

What Does This Do

Tweak delta to relax test conditions on CI.

Motivation

Reduce test flakiness.

Additional Notes

In CI environments, thread scheduling, GC pauses, and CPU contention can easily push timings beyond specified limits.
That is usually results in flaky test with error like:

Condition not satisfied:

delta(timestamps.get(2), timestamps.get(1), 0.1, NANOSECONDS.convert(300, MILLISECONDS))
|     |          |       |          |                        |
false |          |       |          311950274182             300000000
      |          |       [311649930338, 311950274182, 312284536899, 312585003740]
      |          312284536899
      [311649930338, 311950274182, 312284536899, 312585003740]

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD self-assigned this Jan 22, 2026
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD added comp: core Tracer core tag: no release notes Changes to exclude from release notes comp: api Tracer public API labels Jan 22, 2026
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD marked this pull request as ready for review January 22, 2026 22:02
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD requested a review from a team as a code owner January 22, 2026 22:02
@pr-commenter
Copy link

pr-commenter bot commented Jan 22, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/tweak-delta-to-make-test-less-flaky
git_commit_date 1769086485 1769118885
git_commit_sha 3d62379 171265b
release_version 1.59.0-SNAPSHOT~3d62379c90 1.59.0-SNAPSHOT~171265bc0d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1769121196 1769121196
ci_job_id 1371579135 1371579135
ci_pipeline_id 92388343 92388343
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-q96mgrsr 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-q96mgrsr 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 11 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.59.0-SNAPSHOT~171265bc0d, baseline=1.59.0-SNAPSHOT~3d62379c90

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1084260
Total [baseline] (8.75 s) : 0, 8750031
Agent [candidate] (1.084 s) : 0, 1083942
Total [candidate] (8.779 s) : 0, 8779048
section iast
Agent [baseline] (1.224 s) : 0, 1224356
Total [baseline] (9.327 s) : 0, 9326899
Agent [candidate] (1.223 s) : 0, 1222947
Total [candidate] (9.302 s) : 0, 9302428
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.084 s -
Agent iast 1.224 s 140.096 ms (12.9%)
Total tracing 8.75 s -
Total iast 9.327 s 576.868 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.084 s -
Agent iast 1.223 s 139.005 ms (12.8%)
Total tracing 8.779 s -
Total iast 9.302 s 523.38 ms (6.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.59.0-SNAPSHOT~171265bc0d, baseline=1.59.0-SNAPSHOT~3d62379c90

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.179 ms) : 0, 1179
BytebuddyAgent [baseline] (651.8 ms) : 0, 651800
BytebuddyAgent [candidate] (651.151 ms) : 0, 651151
GlobalTracer [baseline] (282.805 ms) : 0, 282805
GlobalTracer [candidate] (283.517 ms) : 0, 283517
AppSec [baseline] (32.484 ms) : 0, 32484
AppSec [candidate] (32.418 ms) : 0, 32418
Debugger [baseline] (65.64 ms) : 0, 65640
Debugger [candidate] (67.035 ms) : 0, 67035
Remote Config [baseline] (624.673 µs) : 0, 625
Remote Config [candidate] (612.624 µs) : 0, 613
Telemetry [baseline] (8.851 ms) : 0, 8851
Telemetry [candidate] (8.807 ms) : 0, 8807
Flare Poller [baseline] (5.335 ms) : 0, 5335
Flare Poller [candidate] (3.848 ms) : 0, 3848
section iast
crashtracking [baseline] (1.178 ms) : 0, 1178
crashtracking [candidate] (1.203 ms) : 0, 1203
BytebuddyAgent [baseline] (792.539 ms) : 0, 792539
BytebuddyAgent [candidate] (791.233 ms) : 0, 791233
GlobalTracer [baseline] (256.765 ms) : 0, 256765
GlobalTracer [candidate] (256.656 ms) : 0, 256656
AppSec [baseline] (33.506 ms) : 0, 33506
AppSec [candidate] (35.158 ms) : 0, 35158
Debugger [baseline] (65.637 ms) : 0, 65637
Debugger [candidate] (63.859 ms) : 0, 63859
Remote Config [baseline] (580.346 µs) : 0, 580
Remote Config [candidate] (559.08 µs) : 0, 559
Telemetry [baseline] (8.379 ms) : 0, 8379
Telemetry [candidate] (8.398 ms) : 0, 8398
Flare Poller [baseline] (3.572 ms) : 0, 3572
Flare Poller [candidate] (3.605 ms) : 0, 3605
IAST [baseline] (26.985 ms) : 0, 26985
IAST [candidate] (26.951 ms) : 0, 26951
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.59.0-SNAPSHOT~171265bc0d, baseline=1.59.0-SNAPSHOT~3d62379c90

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.094 s) : 0, 1093774
Total [baseline] (10.83 s) : 0, 10829926
Agent [candidate] (1.088 s) : 0, 1087823
Total [candidate] (10.827 s) : 0, 10827241
section appsec
Agent [baseline] (1.264 s) : 0, 1263615
Total [baseline] (10.999 s) : 0, 10999015
Agent [candidate] (1.266 s) : 0, 1266349
Total [candidate] (11.06 s) : 0, 11059870
section iast
Agent [baseline] (1.235 s) : 0, 1234780
Total [baseline] (11.243 s) : 0, 11242798
Agent [candidate] (1.227 s) : 0, 1227421
Total [candidate] (11.125 s) : 0, 11124720
section profiling
Agent [baseline] (1.204 s) : 0, 1203592
Total [baseline] (10.892 s) : 0, 10891958
Agent [candidate] (1.205 s) : 0, 1205326
Total [candidate] (10.938 s) : 0, 10937705
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.094 s -
Agent appsec 1.264 s 169.841 ms (15.5%)
Agent iast 1.235 s 141.006 ms (12.9%)
Agent profiling 1.204 s 109.818 ms (10.0%)
Total tracing 10.83 s -
Total appsec 10.999 s 169.09 ms (1.6%)
Total iast 11.243 s 412.873 ms (3.8%)
Total profiling 10.892 s 62.032 ms (0.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.088 s -
Agent appsec 1.266 s 178.526 ms (16.4%)
Agent iast 1.227 s 139.597 ms (12.8%)
Agent profiling 1.205 s 117.503 ms (10.8%)
Total tracing 10.827 s -
Total appsec 11.06 s 232.629 ms (2.1%)
Total iast 11.125 s 297.48 ms (2.7%)
Total profiling 10.938 s 110.464 ms (1.0%) 
gantt
    title petclinic - break down per module: candidate=1.59.0-SNAPSHOT~171265bc0d, baseline=1.59.0-SNAPSHOT~3d62379c90

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.168 ms) : 0, 1168
BytebuddyAgent [baseline] (657.219 ms) : 0, 657219
BytebuddyAgent [candidate] (652.182 ms) : 0, 652182
GlobalTracer [baseline] (285.282 ms) : 0, 285282
GlobalTracer [candidate] (284.457 ms) : 0, 284457
AppSec [baseline] (32.596 ms) : 0, 32596
AppSec [candidate] (32.571 ms) : 0, 32571
Debugger [baseline] (66.817 ms) : 0, 66817
Debugger [candidate] (67.908 ms) : 0, 67908
Remote Config [baseline] (642.089 µs) : 0, 642
Remote Config [candidate] (634.109 µs) : 0, 634
Telemetry [baseline] (8.941 ms) : 0, 8941
Telemetry [candidate] (8.902 ms) : 0, 8902
Flare Poller [baseline] (5.475 ms) : 0, 5475
Flare Poller [candidate] (4.603 ms) : 0, 4603
section appsec
crashtracking [baseline] (1.175 ms) : 0, 1175
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (690.059 ms) : 0, 690059
BytebuddyAgent [candidate] (691.443 ms) : 0, 691443
GlobalTracer [baseline] (258.689 ms) : 0, 258689
GlobalTracer [candidate] (259.47 ms) : 0, 259470
AppSec [baseline] (172.034 ms) : 0, 172034
AppSec [candidate] (170.723 ms) : 0, 170723
Debugger [baseline] (67.95 ms) : 0, 67950
Debugger [candidate] (69.663 ms) : 0, 69663
Remote Config [baseline] (786.857 µs) : 0, 787
Remote Config [candidate] (778.262 µs) : 0, 778
Telemetry [baseline] (9.361 ms) : 0, 9361
Telemetry [candidate] (9.256 ms) : 0, 9256
Flare Poller [baseline] (3.716 ms) : 0, 3716
Flare Poller [candidate] (3.809 ms) : 0, 3809
IAST [baseline] (24.473 ms) : 0, 24473
IAST [candidate] (24.671 ms) : 0, 24671
section iast
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (798.747 ms) : 0, 798747
BytebuddyAgent [candidate] (794.971 ms) : 0, 794971
GlobalTracer [baseline] (258.254 ms) : 0, 258254
GlobalTracer [candidate] (256.434 ms) : 0, 256434
AppSec [baseline] (33.717 ms) : 0, 33717
AppSec [candidate] (34.226 ms) : 0, 34226
Debugger [baseline] (67.316 ms) : 0, 67316
Debugger [candidate] (65.651 ms) : 0, 65651
Remote Config [baseline] (572.269 µs) : 0, 572
Remote Config [candidate] (593.162 µs) : 0, 593
Telemetry [baseline] (8.505 ms) : 0, 8505
Telemetry [candidate] (8.412 ms) : 0, 8412
Flare Poller [baseline] (3.639 ms) : 0, 3639
Flare Poller [candidate] (3.545 ms) : 0, 3545
IAST [baseline] (27.318 ms) : 0, 27318
IAST [candidate] (26.986 ms) : 0, 26986
section profiling
crashtracking [baseline] (1.208 ms) : 0, 1208
crashtracking [candidate] (1.211 ms) : 0, 1211
BytebuddyAgent [baseline] (700.809 ms) : 0, 700809
BytebuddyAgent [candidate] (701.388 ms) : 0, 701388
GlobalTracer [baseline] (221.675 ms) : 0, 221675
GlobalTracer [candidate] (221.43 ms) : 0, 221430
AppSec [baseline] (32.167 ms) : 0, 32167
AppSec [candidate] (32.334 ms) : 0, 32334
Debugger [baseline] (67.74 ms) : 0, 67740
Debugger [candidate] (67.765 ms) : 0, 67765
Remote Config [baseline] (664.788 µs) : 0, 665
Remote Config [candidate] (664.044 µs) : 0, 664
Telemetry [baseline] (9.032 ms) : 0, 9032
Telemetry [candidate] (9.031 ms) : 0, 9031
Flare Poller [baseline] (3.752 ms) : 0, 3752
Flare Poller [candidate] (3.825 ms) : 0, 3825
ProfilingAgent [baseline] (96.833 ms) : 0, 96833
ProfilingAgent [candidate] (97.808 ms) : 0, 97808
Profiling [baseline] (97.411 ms) : 0, 97411
Profiling [candidate] (98.396 ms) : 0, 98396
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/tweak-delta-to-make-test-less-flaky
git_commit_date 1769086485 1769118885
git_commit_sha 3d62379 171265b
release_version 1.59.0-SNAPSHOT~3d62379c90 1.59.0-SNAPSHOT~171265bc0d
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1769121700 1769121700
ci_job_id 1371579136 1371579136
ci_pipeline_id 92388343 92388343
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-aqljlpj1 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-aqljlpj1 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 3 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:tracing:high_load worse
[+29.698µs; +106.669µs] or [+2.014%; +7.233%]		 unstable
[+60.151µs; +732.240µs] or [+1.519%; +18.493%]		 unstable
[-437.869op/s; +123.432op/s] or [-17.204%; +4.850%]		 1.543ms 4.356ms 2387.906op/s 1.475ms 3.959ms 2545.125op/s
scenario:load:insecure-bank:iast:high_load worse
[+115.116µs; +207.133µs] or [+4.768%; +8.580%]		 worse
[+165.288µs; +507.669µs] or [+2.331%; +7.160%]		 unstable
[-227.577op/s; +77.202op/s] or [-15.437%; +5.237%]		 2.575ms 7.427ms 1399.031op/s 2.414ms 7.091ms 1474.219op/s
scenario:load:petclinic:profiling:high_load better
[-1.560ms; -0.481ms] or [-7.846%; -2.419%]		 same
[-1291.418µs; +795.649µs] or [-4.147%; +2.555%]		 unstable
[-16.368op/s; +35.681op/s] or [-6.997%; +15.252%]		 18.859ms 30.891ms 243.594op/s 19.879ms 31.139ms 233.938op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~171265bc0d, baseline=1.59.0-SNAPSHOT~3d62379c90
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.198 ms) : 1186, 1210
.   : milestone, 1198,
iast (3.102 ms) : 3061, 3142
.   : milestone, 3102,
iast_FULL (5.684 ms) : 5629, 5739
.   : milestone, 5684,
iast_GLOBAL (3.495 ms) : 3448, 3542
.   : milestone, 3495,
profiling (1.992 ms) : 1975, 2009
.   : milestone, 1992,
tracing (1.767 ms) : 1752, 1781
.   : milestone, 1767,
section candidate
no_agent (1.186 ms) : 1175, 1197
.   : milestone, 1186,
iast (3.272 ms) : 3228, 3315
.   : milestone, 3272,
iast_FULL (5.762 ms) : 5704, 5819
.   : milestone, 5762,
iast_GLOBAL (3.427 ms) : 3380, 3475
.   : milestone, 3427,
profiling (2.041 ms) : 2023, 2059
.   : milestone, 2041,
tracing (1.886 ms) : 1868, 1904
.   : milestone, 1886,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.198 ms [1.186 ms, 1.21 ms] -
iast 3.102 ms [3.061 ms, 3.142 ms] 1.903 ms (158.9%)
iast_FULL 5.684 ms [5.629 ms, 5.739 ms] 4.486 ms (374.4%)
iast_GLOBAL 3.495 ms [3.448 ms, 3.542 ms] 2.297 ms (191.7%)
profiling 1.992 ms [1.975 ms, 2.009 ms] 793.724 µs (66.2%)
tracing 1.767 ms [1.752 ms, 1.781 ms] 568.408 µs (47.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.186 ms [1.175 ms, 1.197 ms] -
iast 3.272 ms [3.228 ms, 3.315 ms] 2.086 ms (175.9%)
iast_FULL 5.762 ms [5.704 ms, 5.819 ms] 4.576 ms (385.8%)
iast_GLOBAL 3.427 ms [3.38 ms, 3.475 ms] 2.242 ms (189.0%)
profiling 2.041 ms [2.023 ms, 2.059 ms] 854.917 µs (72.1%)
tracing 1.886 ms [1.868 ms, 1.904 ms] 700.084 µs (59.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~171265bc0d, baseline=1.59.0-SNAPSHOT~3d62379c90
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.963 ms) : 17781, 18144
.   : milestone, 17963,
appsec (18.987 ms) : 18793, 19180
.   : milestone, 18987,
code_origins (17.621 ms) : 17447, 17794
.   : milestone, 17621,
iast (17.793 ms) : 17616, 17970
.   : milestone, 17793,
profiling (19.955 ms) : 19755, 20156
.   : milestone, 19955,
tracing (17.795 ms) : 17617, 17973
.   : milestone, 17795,
section candidate
no_agent (18.228 ms) : 18042, 18415
.   : milestone, 18228,
appsec (18.731 ms) : 18541, 18922
.   : milestone, 18731,
code_origins (17.754 ms) : 17578, 17931
.   : milestone, 17754,
iast (17.514 ms) : 17343, 17685
.   : milestone, 17514,
profiling (19.159 ms) : 18962, 19356
.   : milestone, 19159,
tracing (17.885 ms) : 17708, 18061
.   : milestone, 17885,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.963 ms [17.781 ms, 18.144 ms] -
appsec 18.987 ms [18.793 ms, 19.18 ms] 1.024 ms (5.7%)
code_origins 17.621 ms [17.447 ms, 17.794 ms] -341.842 µs (-1.9%)
iast 17.793 ms [17.616 ms, 17.97 ms] -169.336 µs (-0.9%)
profiling 19.955 ms [19.755 ms, 20.156 ms] 1.993 ms (11.1%)
tracing 17.795 ms [17.617 ms, 17.973 ms] -167.868 µs (-0.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.228 ms [18.042 ms, 18.415 ms] -
appsec 18.731 ms [18.541 ms, 18.922 ms] 503.374 µs (2.8%)
code_origins 17.754 ms [17.578 ms, 17.931 ms] -473.644 µs (-2.6%)
iast 17.514 ms [17.343 ms, 17.685 ms] -713.883 µs (-3.9%)
profiling 19.159 ms [18.962 ms, 19.356 ms] 930.935 µs (5.1%)
tracing 17.885 ms [17.708 ms, 18.061 ms] -343.254 µs (-1.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/tweak-delta-to-make-test-less-flaky
git_commit_date 1769086485 1769118885
git_commit_sha 3d62379 171265b
release_version 1.59.0-SNAPSHOT~3d62379c90 1.59.0-SNAPSHOT~171265bc0d
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1769121609 1769121609
ci_job_id 1371579137 1371579137
ci_pipeline_id 92388343 92388343
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-vryxj4ri 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-vryxj4ri 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~171265bc0d, baseline=1.59.0-SNAPSHOT~3d62379c90
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.633 s) : 15633000, 15633000
.   : milestone, 15633000,
appsec (14.764 s) : 14764000, 14764000
.   : milestone, 14764000,
iast (17.88 s) : 17880000, 17880000
.   : milestone, 17880000,
iast_GLOBAL (17.744 s) : 17744000, 17744000
.   : milestone, 17744000,
profiling (15.142 s) : 15142000, 15142000
.   : milestone, 15142000,
tracing (14.908 s) : 14908000, 14908000
.   : milestone, 14908000,
section candidate
no_agent (15.326 s) : 15326000, 15326000
.   : milestone, 15326000,
appsec (14.906 s) : 14906000, 14906000
.   : milestone, 14906000,
iast (17.973 s) : 17973000, 17973000
.   : milestone, 17973000,
iast_GLOBAL (17.925 s) : 17925000, 17925000
.   : milestone, 17925000,
profiling (14.915 s) : 14915000, 14915000
.   : milestone, 14915000,
tracing (14.897 s) : 14897000, 14897000
.   : milestone, 14897000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.633 s [15.633 s, 15.633 s] -
appsec 14.764 s [14.764 s, 14.764 s] -869.0 ms (-5.6%)
iast 17.88 s [17.88 s, 17.88 s] 2.247 s (14.4%)
iast_GLOBAL 17.744 s [17.744 s, 17.744 s] 2.111 s (13.5%)
profiling 15.142 s [15.142 s, 15.142 s] -491.0 ms (-3.1%)
tracing 14.908 s [14.908 s, 14.908 s] -725.0 ms (-4.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.326 s [15.326 s, 15.326 s] -
appsec 14.906 s [14.906 s, 14.906 s] -420.0 ms (-2.7%)
iast 17.973 s [17.973 s, 17.973 s] 2.647 s (17.3%)
iast_GLOBAL 17.925 s [17.925 s, 17.925 s] 2.599 s (17.0%)
profiling 14.915 s [14.915 s, 14.915 s] -411.0 ms (-2.7%)
tracing 14.897 s [14.897 s, 14.897 s] -429.0 ms (-2.8%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~171265bc0d, baseline=1.59.0-SNAPSHOT~3d62379c90
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1458, 1481
.   : milestone, 1470,
appsec (3.731 ms) : 3511, 3951
.   : milestone, 3731,
iast (2.2 ms) : 2135, 2264
.   : milestone, 2200,
iast_GLOBAL (2.245 ms) : 2180, 2311
.   : milestone, 2245,
profiling (2.067 ms) : 2015, 2120
.   : milestone, 2067,
tracing (2.052 ms) : 2001, 2104
.   : milestone, 2052,
section candidate
no_agent (1.479 ms) : 1467, 1491
.   : milestone, 1479,
appsec (3.65 ms) : 3436, 3864
.   : milestone, 3650,
iast (2.21 ms) : 2145, 2275
.   : milestone, 2210,
iast_GLOBAL (2.257 ms) : 2191, 2323
.   : milestone, 2257,
profiling (2.088 ms) : 2034, 2142
.   : milestone, 2088,
tracing (2.055 ms) : 2003, 2106
.   : milestone, 2055,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.458 ms, 1.481 ms] -
appsec 3.731 ms [3.511 ms, 3.951 ms] 2.262 ms (153.9%)
iast 2.2 ms [2.135 ms, 2.264 ms] 729.809 µs (49.7%)
iast_GLOBAL 2.245 ms [2.18 ms, 2.311 ms] 775.593 µs (52.8%)
profiling 2.067 ms [2.015 ms, 2.12 ms] 597.493 µs (40.7%)
tracing 2.052 ms [2.001 ms, 2.104 ms] 582.611 µs (39.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.479 ms [1.467 ms, 1.491 ms] -
appsec 3.65 ms [3.436 ms, 3.864 ms] 2.171 ms (146.8%)
iast 2.21 ms [2.145 ms, 2.275 ms] 731.093 µs (49.4%)
iast_GLOBAL 2.257 ms [2.191 ms, 2.323 ms] 777.772 µs (52.6%)
profiling 2.088 ms [2.034 ms, 2.142 ms] 608.878 µs (41.2%)
tracing 2.055 ms [2.003 ms, 2.106 ms] 575.587 µs (38.9%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jpbempel jpbempel Awaiting requested review from jpbempel

@amarziali amarziali Awaiting requested review from amarziali amarziali is a code owner automatically assigned from DataDog/apm-java

At least 1 approving review is required to merge this pull request.

Assignees

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD

Labels

comp: api Tracer public API comp: core Tracer core tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@AlexeyKuznetsov-DD