Skip to content

Conversation

@zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Dec 30, 2025

What problem does this PR solve?

Problem Summary:

  1. support iceberg external table in analyze
  2. fix SHOW QUEUED ANALYZE JOBS could running normal.
mysql> SHOW QUEUED ANALYZE JOBS;
+---------------------------+---------------+-------------------------+---------------------------------------------------------------+----------+
| catalog_name              | db_name       | tbl_name                | col_list                                                      | priority |
+---------------------------+---------------+-------------------------+---------------------------------------------------------------+----------+
| test_iceberg_with_mapping | multi_catalog | orc_partitioned_columns | orc_partitioned_columns:t_int,orc_partitioned_columns:t_float | HIGH     |
+---------------------------+---------------+-------------------------+---------------------------------------------------------------+----------+

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34232 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e7265dec17377d142570c6b66ea299c9e535449a, data reload: false

------ Round 1 ----------------------------------
q1	17610	4200	4060	4060
q2	2031	346	233	233
q3	10193	1314	749	749
q4	10211	824	321	321
q5	7525	2108	1866	1866
q6	194	172	138	138
q7	935	800	664	664
q8	9257	1397	1108	1108
q9	6866	5145	5143	5143
q10	6790	1791	1404	1404
q11	532	308	289	289
q12	707	732	588	588
q13	17789	3816	3071	3071
q14	308	300	287	287
q15	600	512	502	502
q16	679	681	629	629
q17	695	716	642	642
q18	7491	7342	7161	7161
q19	1096	955	612	612
q20	394	364	252	252
q21	4133	3806	3523	3523
q22	1082	1024	990	990
Total cold run time: 107118 ms
Total hot run time: 34232 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4084	4039	4048	4039
q2	336	398	319	319
q3	2077	2611	2215	2215
q4	1333	1726	1307	1307
q5	4085	4008	4668	4008
q6	230	181	141	141
q7	2021	1922	1782	1782
q8	2577	2308	2452	2308
q9	7247	7050	7084	7050
q10	2456	2785	2327	2327
q11	576	507	448	448
q12	747	747	623	623
q13	3740	3995	3557	3557
q14	287	308	278	278
q15	540	503	493	493
q16	642	697	623	623
q17	1145	1302	1345	1302
q18	7975	7809	7885	7809
q19	900	890	904	890
q20	1949	2098	1894	1894
q21	4520	4211	4077	4077
q22	1099	1068	1006	1006
Total cold run time: 50566 ms
Total hot run time: 48496 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174729 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e7265dec17377d142570c6b66ea299c9e535449a, data reload: false

query5	4854	569	422	422
query6	329	213	210	210
query7	4201	457	266	266
query8	334	245	235	235
query9	8782	2578	2605	2578
query10	564	377	320	320
query11	15331	15122	14805	14805
query12	178	116	115	115
query13	1252	506	391	391
query14	6157	2927	2691	2691
query14_1	2616	2581	2611	2581
query15	203	188	175	175
query16	1002	460	376	376
query17	1070	680	593	593
query18	2494	449	343	343
query19	230	226	198	198
query20	124	119	116	116
query21	213	148	129	129
query22	3960	4145	4117	4117
query23	15946	15770	15214	15214
query23_1	15445	15499	15489	15489
query24	7368	1609	1208	1208
query24_1	1230	1200	1237	1200
query25	565	485	434	434
query26	1261	267	157	157
query27	2759	453	298	298
query28	4533	2185	2159	2159
query29	794	550	451	451
query30	312	247	210	210
query31	820	628	561	561
query32	80	68	70	68
query33	549	330	292	292
query34	900	877	522	522
query35	772	767	734	734
query36	863	876	795	795
query37	128	83	75	75
query38	2679	2732	2719	2719
query39	776	753	731	731
query39_1	701	710	710	710
query40	214	128	111	111
query41	66	64	64	64
query42	103	103	104	103
query43	475	465	416	416
query44	1322	765	749	749
query45	185	184	172	172
query46	879	957	599	599
query47	1349	1406	1319	1319
query48	319	320	248	248
query49	604	407	313	313
query50	630	274	216	216
query51	3783	3962	3758	3758
query52	99	106	100	100
query53	301	320	271	271
query54	276	268	236	236
query55	76	75	70	70
query56	272	282	292	282
query57	1032	975	906	906
query58	263	242	247	242
query59	1947	2072	1984	1984
query60	318	306	316	306
query61	157	150	155	150
query62	393	364	322	322
query63	321	268	268	268
query64	4945	1316	972	972
query65	3817	3709	3672	3672
query66	1440	423	313	313
query67	15151	15542	15854	15542
query68	7807	998	722	722
query69	491	356	296	296
query70	1068	950	935	935
query71	377	298	261	261
query72	6105	4962	5009	4962
query73	708	637	312	312
query74	8783	8840	8557	8557
query75	2920	2865	2496	2496
query76	3951	1044	648	648
query77	515	362	281	281
query78	9767	9837	9194	9194
query79	1572	910	600	600
query80	1344	581	482	482
query81	526	272	228	228
query82	415	139	109	109
query83	263	257	231	231
query84	257	125	105	105
query85	905	509	463	463
query86	351	322	325	322
query87	2845	2862	2734	2734
query88	3231	2296	2304	2296
query89	394	354	320	320
query90	1973	154	146	146
query91	172	172	140	140
query92	70	68	67	67
query93	1122	903	561	561
query94	652	331	284	284
query95	571	322	353	322
query96	597	449	209	209
query97	2320	2353	2257	2257
query98	236	208	200	200
query99	629	590	500	500
Total cold run time: 255256 ms
Total hot run time: 174729 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 26.81 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e7265dec17377d142570c6b66ea299c9e535449a, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.05	0.04
query3	0.26	0.09	0.08
query4	1.60	0.11	0.12
query5	0.28	0.26	0.25
query6	1.14	0.66	0.65
query7	0.03	0.02	0.03
query8	0.05	0.04	0.05
query9	0.55	0.51	0.50
query10	0.56	0.53	0.55
query11	0.15	0.11	0.12
query12	0.16	0.13	0.12
query13	0.60	0.59	0.60
query14	0.99	0.98	0.99
query15	0.80	0.78	0.79
query16	0.39	0.39	0.40
query17	1.00	0.99	1.04
query18	0.23	0.21	0.21
query19	1.87	1.78	1.82
query20	0.01	0.01	0.02
query21	15.44	0.29	0.14
query22	4.95	0.06	0.05
query23	16.09	0.29	0.11
query24	1.52	0.87	0.31
query25	0.11	0.07	0.05
query26	0.14	0.12	0.12
query27	0.07	0.06	0.06
query28	4.86	1.04	0.87
query29	12.59	4.00	3.23
query30	0.28	0.14	0.13
query31	2.82	0.61	0.38
query32	3.23	0.55	0.46
query33	2.89	3.08	3.04
query34	16.72	5.07	4.42
query35	4.44	4.48	4.38
query36	0.64	0.50	0.50
query37	0.11	0.06	0.06
query38	0.07	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.14	0.14
query41	0.09	0.04	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 98.18 s
Total hot run time: 26.81 s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 35488 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e7265dec17377d142570c6b66ea299c9e535449a, data reload: false

------ Round 1 ----------------------------------
q1	17606	4269	4063	4063
q2	2013	352	260	260
q3	10179	1279	726	726
q4	10219	887	326	326
q5	7488	2126	1899	1899
q6	187	169	135	135
q7	930	791	660	660
q8	9281	1388	1146	1146
q9	6911	5207	5190	5190
q10	6803	1812	1406	1406
q11	512	302	295	295
q12	718	709	603	603
q13	17798	3827	3086	3086
q14	303	287	268	268
q15	587	523	509	509
q16	711	700	647	647
q17	706	745	622	622
q18	7695	7515	7972	7515
q19	1803	1025	646	646
q20	436	402	259	259
q21	4624	4274	4181	4181
q22	1167	1133	1046	1046
Total cold run time: 108677 ms
Total hot run time: 35488 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4327	4201	4308	4201
q2	348	402	329	329
q3	2346	2804	2404	2404
q4	1473	2052	1392	1392
q5	4399	4384	4338	4338
q6	206	181	150	150
q7	2002	1933	1825	1825
q8	2557	2337	2271	2271
q9	7200	7170	7410	7170
q10	2440	2743	2171	2171
q11	507	448	428	428
q12	671	700	565	565
q13	3345	3847	3133	3133
q14	276	281	267	267
q15	527	488	481	481
q16	613	631	604	604
q17	1091	1174	1236	1174
q18	7402	7333	7313	7313
q19	853	821	863	821
q20	1883	1990	1796	1796
q21	4633	4317	4158	4158
q22	1141	1028	1035	1028
Total cold run time: 50240 ms
Total hot run time: 48019 ms

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 4.17% (1/24) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

TPC-DS: Total hot run time: 175707 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e7265dec17377d142570c6b66ea299c9e535449a, data reload: false

query5	4845	587	411	411
query6	331	227	228	227
query7	4206	456	263	263
query8	336	249	235	235
query9	8763	2609	2611	2609
query10	527	375	323	323
query11	15078	15224	14951	14951
query12	170	113	110	110
query13	1243	488	408	408
query14	6284	2964	2703	2703
query14_1	2623	2742	2637	2637
query15	191	186	171	171
query16	977	470	468	468
query17	1038	674	581	581
query18	2511	424	324	324
query19	217	215	185	185
query20	118	115	110	110
query21	213	139	114	114
query22	4088	3918	3889	3889
query23	16383	15753	15588	15588
query23_1	15445	15517	15525	15517
query24	7417	1586	1222	1222
query24_1	1215	1196	1181	1181
query25	551	446	412	412
query26	1254	262	160	160
query27	2753	451	289	289
query28	4554	2190	2173	2173
query29	763	519	416	416
query30	321	241	219	219
query31	813	643	535	535
query32	79	67	71	67
query33	532	326	274	274
query34	878	873	532	532
query35	750	799	724	724
query36	873	875	806	806
query37	123	94	76	76
query38	2747	2697	2661	2661
query39	768	769	721	721
query39_1	726	719	719	719
query40	233	136	115	115
query41	67	63	61	61
query42	104	100	104	100
query43	460	437	416	416
query44	1322	754	760	754
query45	186	181	173	173
query46	864	956	608	608
query47	1385	1475	1340	1340
query48	313	318	248	248
query49	625	428	334	334
query50	643	284	206	206
query51	3803	3794	3807	3794
query52	103	111	100	100
query53	290	331	275	275
query54	282	261	277	261
query55	77	80	74	74
query56	288	296	293	293
query57	1024	1065	908	908
query58	270	248	241	241
query59	2128	2248	2085	2085
query60	319	317	287	287
query61	163	161	164	161
query62	412	373	340	340
query63	296	261	279	261
query64	4922	1313	1078	1078
query65	3811	3673	3664	3664
query66	1456	460	339	339
query67	15464	15553	15732	15553
query68	2794	1010	762	762
query69	450	369	320	320
query70	990	862	909	862
query71	312	296	290	290
query72	6032	5021	5071	5021
query73	507	597	311	311
query74	8826	8784	8631	8631
query75	2828	2883	2506	2506
query76	2857	1044	650	650
query77	342	361	287	287
query78	9897	10085	9241	9241
query79	1167	904	600	600
query80	1264	556	488	488
query81	538	261	245	245
query82	995	143	106	106
query83	342	257	242	242
query84	262	119	97	97
query85	930	517	453	453
query86	405	294	317	294
query87	2916	2843	2777	2777
query88	3311	2269	2258	2258
query89	381	352	319	319
query90	1936	157	149	149
query91	177	163	142	142
query92	92	65	61	61
query93	1072	900	570	570
query94	640	321	295	295
query95	566	392	306	306
query96	584	470	207	207
query97	2357	2381	2289	2289
query98	213	209	192	192
query99	588	590	517	517
Total cold run time: 249918 ms
Total hot run time: 175707 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 26.96 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e7265dec17377d142570c6b66ea299c9e535449a, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.05	0.06
query3	0.26	0.08	0.08
query4	1.60	0.11	0.11
query5	0.29	0.25	0.27
query6	1.14	0.66	0.67
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.56	0.50	0.50
query10	0.54	0.54	0.54
query11	0.15	0.11	0.12
query12	0.16	0.13	0.12
query13	0.63	0.60	0.58
query14	0.99	0.99	0.97
query15	0.80	0.80	0.79
query16	0.43	0.42	0.43
query17	1.02	1.05	1.06
query18	0.23	0.22	0.21
query19	1.92	1.81	1.85
query20	0.02	0.01	0.01
query21	15.43	0.26	0.13
query22	4.92	0.05	0.04
query23	16.19	0.29	0.10
query24	1.74	0.28	0.26
query25	0.10	0.09	0.05
query26	0.14	0.13	0.14
query27	0.07	0.06	0.06
query28	3.47	1.06	0.88
query29	12.57	3.90	3.13
query30	0.28	0.13	0.11
query31	2.81	0.63	0.37
query32	3.22	0.55	0.48
query33	3.03	3.00	3.05
query34	16.60	5.06	4.54
query35	4.46	4.48	4.51
query36	0.64	0.50	0.48
query37	0.10	0.06	0.06
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 97.18 s
Total hot run time: 26.96 s

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 12.50% (3/24) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants