TableBench: A Comprehensive and Complex Benchmark for Table Question Answering

PaperCodeTableBenchTableInstructTableLLMsHome
DP
TCoT
SCoT
PoT
                    
                

                Model
                
                
            
            
            
            
            
        
            

                    
                

                Rank
                
                
            
            
            
            
            
        
            

                    
                

                Fact Checking
                
                
            
            
            
            
            
        
            

                    
                

                Numerical Reasoning
                
                
            
            
            
            
            
        
            

                    
                

                Data Analysis
                
                
            
            
            
            
            
        
            

                    
                

                Visualization
                
                
            
            
            
            
            
        
            

                    
                

                Overall
                
                
            
            
            
            
            
        
            
GPT-4o
Deepseek-Chat-V2
GPT-4-Turbo
Deepseek-Coder-V2
Llama3.1-70B-Instruct
Qwen2-72B-Instruct
Yi-Large
GLM-4
Llama3-70B-Chat
Qwen1.5-110B-Chat
Qwen-Max
Qwen1.5-72B-Chat
TableLLM-Deepseek-Coder-7B
GPT-3.5-Turbo
Llama3-8B-Chat
TableLLM-Llama3.1-8B
TableLLM-Qwen2-7B
TableLLM-Llama3-8B
TableLLM-CodeQwen-7B
Mixtral-8x7B-Instruct
Llama3.1-8B-Instruct
CodeLlama-34B-Instruct
Qwen2-7B-Instruct
WizardLM-13B
Qwen1.5-32B-Chat
Mistral-7B-Instruct
Llama2-13B-Chat
Qwen1.5-14B-Chat
CodeLlama-7B-Instruct
Llama2-7B-Chat
CodeQwen1.5-7B-Chat
Qwen1.5-7B-Chat
Gemma-7B-Instruct
Deepseek-Coder-7B-Instruct
MAP-Neo-7B-Instruct
StructLM-7B
StructLM-13B
Deepseek-Coder-33B-Instruct
CodeLlama-70b-Instruct
StructLM-34B
1
74.29
42.73
34.36
38
42.73
2
68.82
40.87
35.71
20
40.65
3
72.97
40.17
32.34
34
40.38
4
70.25
31.48
31.07
26
35.21
5
72.31
27.6
31.14
22
33.63
6
71.11
27.32
30.9
10
32.52
7
70.26
25.01
29.74
37.5
32.43
8
70.63
24.99
31.19
6
31.23
9
72.24
23.09
31.58
12
30.91
10
72.37
22.28
28.28
18
29.72
11
67.01
22.28
28.55
22.92
29.63
12
75.34
21.1
28.02
0
28.45
13
67.32
17.14
29.86
26
27.98
14
56.5
20.52
31.84
0
27.75
15
70.79
18.55
28.93
0
27.28
16
63.68
18.01
28.32
24
27.19
17
65.25
16.7
28.94
24
27.14
18
66.03
16.59
28.02
28
26.93
19
65.02
13.48
29.82
26
26.08
20
66.24
15.81
25.99
10
24.98
21
64.13
13.28
27.15
2
23.47
22
58.97
12.24
24.83
2
21.6
23
53.76
15.75
21.51
0
21.23
24
52.35
11.48
25.5
0
20.8
25
42.32
15.6
21.46
4
20.21
26
50.15
9.32
24.55
0
19.15
27
46.49
9.72
23.25
2
18.58
28
34.8
11.08
22.9
2
17.76
29
42.5
6.1
24.88
0
17.01
30
41.14
7.43
23.78
0
16.98
31
32.01
7.08
25.86
0
16.76
32
26.47
8.04
24.13
0
15.84
33
29.69
6.44
22.21
2
14.82
34
21.6
4.75
22.17
14
13.82
35
22.43
7.17
18.21
0
12.66
36
34.64
5.79
14.37
2
12.06
37
27.76
6.74
14.3
0
11.52
38
28.38
5.09
10.13
8
9.74
39
13.37
1.64
7.69
10
5.73
40
0
0.5
0.99
0
0.6

                
                
                    
                    to
                    
                    of
                    
                
                

                        Page
                        
                        of
                        
                    

            