FlagGems Experimental Operators#

This document lists all experimental operators in FlagGems that have achieved an average speedup of 0.8x or higher compared to PyTorch implementations.

Performance Overview#

Total Operators: 142
Average Speedup Range: 0.81x - 7.23x
Test Environment: Hopper GPU
Filtering Criteria: Average speedup ≥ 0.8x

Operators by Performance#

Rank	Operator	Avg Speedup	Category
1	`_safe_softmax`	7.23x 🏆	Internal
2	`digamma_`	2.41x 🏆	Math
3	`zero`	1.85x ✅	Other
4	`relu`	1.79x ✅	Activation
5	`mse_loss`	1.64x ✅	Loss
6	`masked_select`	1.47x ✅	Other
7	`masked_scatter`	1.44x ✅	Other
8	`eye`	1.43x ✅	Other
9	`t_copy`	1.41x ✅	Shape
10	`trace`	1.40x ✅	Math
11	`i0_`	1.37x ✅	Math
12	`zeros_like`	1.32x ✅	Other
13	`diag`	1.27x ✅	Other
14	`lift_fresh_copy`	1.24x ✅	Other
15	`alias_copy`	1.23x ✅	Other
16	`pixel_unshuffle`	1.20x 📈	Vision
17	`triu`	1.18x 📈	Shape
18	`rrelu_with_noise_backward`	1.17x 📈	Activation
19	`glu`	1.17x 📈	Activation
20	`tril`	1.16x 📈	Shape
21	`silu_`	1.16x 📈	Activation
22	`asinh_`	1.14x 📈	Math
23	`mv`	1.14x 📈	Linear Algebra
24	`arcsinh_`	1.13x 📈	Math
25	`pixel_shuffle`	1.12x 📈	Vision
26	`replication_pad3d`	1.11x 📈	Padding
27	`_upsample_nearest_exact1d`	1.11x 📈	Vision
28	`i0`	1.11x 📈	Math
29	`softplus`	1.10x 📈	Activation
30	`selu_`	1.10x 📈	Activation
31	`upsample_nearest1d`	1.10x 📈	Vision
32	`special_i1`	1.09x 📈	Math
33	`selu`	1.09x 📈	Activation
34	`amin`	1.09x 📈	Math
35	`sinh_`	1.09x 📈	Math
36	`logit_`	1.08x 📈	Math
37	`upsample_nearest3d`	1.07x 📈	Vision
38	`im2col`	1.06x 📈	Vision
39	`reflection_pad1d`	1.06x 📈	Padding
40	`elu`	1.06x 📈	Activation
41	`arctanh_`	1.05x 📈	Math
42	`sigmoid`	1.05x 📈	Activation
43	`replication_pad1d`	1.04x 📈	Padding
44	`silu`	1.04x 📈	Activation
45	`sigmoid_`	1.04x 📈	Activation
46	`addcdiv`	1.04x 📈	Arithmetic
47	`sinc_`	1.03x 📈	Math
48	`relu6`	1.03x 📈	Activation
49	`hardtanh`	1.03x 📈	Activation
50	`hardtanh_`	1.03x 📈	Activation
51	`hardswish_`	1.03x 📈	Activation
52	`reciprocal_`	1.03x 📈	Math
53	`sinc`	1.03x 📈	Math
54	`hardsigmoid`	1.03x 📈	Activation
55	`logaddexp2`	1.02x 📈	Math
56	`logit`	1.02x 📈	Math
57	`arctanh`	1.02x 📈	Math
58	`logaddexp`	1.02x 📈	Math
59	`cosh_`	1.02x 📈	Math
60	`special_xlog1py`	1.02x 📈	Math
61	`celu`	1.02x 📈	Activation
62	`hardsigmoid_`	1.02x 📈	Activation
63	`arcsinh`	1.02x 📈	Math
64	`sign`	1.02x 📈	Math
65	`absolute_`	1.01x 📈	Math
66	`_adaptive_avg_pool3d`	1.01x 📈	Vision
67	`special_i0e`	1.01x 📈	Math
68	`cos_`	1.01x 📈	Math
69	`deg2rad_`	1.01x 📈	Math
70	`floor_`	1.01x 📈	Math
71	`negative`	1.01x 📈	Math
72	`xlogy`	1.01x 📈	Math
73	`exp2`	1.01x 📈	Math
74	`exp_`	1.00x 📈	Math
75	`fix`	1.00x 📈	Math
76	`xlogy_`	1.00x 📈	Math
77	`absolute`	1.00x 📈	Math
78	`prelu`	1.00x 📈	Activation
79	`hypot`	1.00x 📈	Math
80	`rad2deg_`	1.00x 📈	Math
81	`smooth_l1_loss`	1.00x 📈	Loss
82	`deg2rad`	1.00x 📈	Math
83	`log_`	1.00x 📈	Math
84	`sgn_`	1.00x 📈	Math
85	`sin_`	1.00x 📈	Math
86	`heaviside`	1.00x 📈	Math
87	`logical_xor_`	1.00x 📈	Other
88	`trunc`	1.00x 📈	Math
89	`heaviside_`	1.00x 📈	Math
90	`hardshrink`	1.00x 📈	Activation
91	`huber_loss`	1.00x 📈	Loss
92	`threshold_`	1.00x 📈	Activation
93	`addcmul_`	1.00x 📈	Arithmetic
94	`neg_`	1.00x 📈	Math
95	`hypot_`	1.00x 📈	Math
96	`leaky_relu`	1.00x 📈	Activation
97	`fmin`	1.00x 📈	Math
98	`erfinv`	1.00x 📈	Math
99	`log1p_`	1.00x 📈	Math
100	`frac`	1.00x ⚡	Math
101	`_functional_sym_constrain_range_for_size`	1.00x ⚡	Internal
102	`expand`	1.00x ⚡	Shape
103	`lift`	1.00x ⚡	Other
104	`unsqueeze`	1.00x ⚡	Shape
105	`_unsafe_view`	1.00x ⚡	Internal
106	`softshrink`	1.00x ⚡	Activation
107	`log2_`	1.00x ⚡	Math
108	`permute`	1.00x ⚡	Shape
109	`leaky_relu_`	1.00x ⚡	Activation
110	`atanh_`	1.00x ⚡	Math
111	`permute_copy`	1.00x ⚡	Shape
112	`fft_ifftshift`	1.00x ⚡	Other
113	`copy_`	1.00x ⚡	Other
114	`fix_`	1.00x ⚡	Math
115	`slice_scatter`	0.99x ⚡	Other
116	`exp2_`	0.99x ⚡	Math
117	`rsqrt_`	0.99x ⚡	Math
118	`threshold`	0.98x ⚡	Activation
119	`reciprocal`	0.97x ⚡	Math
120	`maximum`	0.97x ⚡	Arithmetic
121	`abs`	0.96x ⚡	Math
122	`arccosh`	0.96x ⚡	Math
123	`multiply`	0.95x ⚡	Arithmetic
124	`margin_ranking_loss`	0.95x ⚡	Loss
125	`celu_`	0.92x ⚡	Activation
126	`hardswish`	0.91x ⚡	Activation
127	`soft_margin_loss`	0.90x ⚡	Loss
128	`replication_pad2d`	0.90x ⚡	Padding
129	`unsqueeze_copy`	0.89x ⚡	Shape
130	`native_dropout_backward`	0.89x ⚡	Other
131	`slice_backward`	0.88x ⚡	Other
132	`relu_`	0.86x ⚡	Activation
133	`negative_`	0.86x ⚡	Math
134	`abs_`	0.86x ⚡	Math
135	`take`	0.86x ⚡	Other
136	`sgn`	0.86x ⚡	Math
137	`erf_`	0.82x ⚡	Math
138	`gelu_`	0.82x ⚡	Activation
139	`erfinv_`	0.82x ⚡	Math
140	`_log_softmax_backward_data`	0.82x ⚡	Internal
141	`log10_`	0.81x ⚡	Math
142	`rmsnorm`	special ⚡	Normalization

Legend:

🏆 Outstanding: Speedup ≥ 2.0x
✅ Excellent: Speedup ≥ 1.5x
📈 Good: Speedup ≥ 1.0x
⚡ Decent: Speedup ≥ 0.8x

Categories#

Activation: Activation functions (ReLU, GELU, Sigmoid, etc.)
Arithmetic: Basic arithmetic operations (add, mul, div, etc.)
Comparison: Comparison operations (eq, ne, gt, lt, etc.)
Internal: Internal/utility operations
Linear Algebra: Matrix operations (matmul, mv, etc.)
Loss: Loss functions (MSE, Cross-Entropy, etc.)
Math: Mathematical functions (sin, cos, exp, log, etc.)
NLP: Natural language processing operations
Other: Miscellaneous operations
Padding: Padding operations (reflection_pad, replication_pad, etc.)
Shape: Shape manipulation operations
Vision: Computer vision operations

Notes#

All operators have passed accuracy tests
Performance measured on Hopper GPU with various input shapes
Speedup calculated as: PyTorch_time / FlagGems_time
Higher values indicate better performance