Speed up nmod_poly_evaluate_nmod asymptotically

[fredrik-johansson]

Rectangular splitting makes nmod_poly_evaluate_nmod as fast as dot products asymptotically. On Zen 3 this is 14x faster for small moduli, 3x faster for 62/63-bit moduli, and 7x faster for 64-bit moduli.

Also, algorithm selection is moved to the underscore method.

The crossover at length 50 could be fine-tuned.

This is might obsolete some of the optimizations proposed in #2492 (others may be complementary).

unit: all measurements in c/l
profiled: interface | horner | precomp | precomp_lazy | rectangular
nbits = 16
   len 1	7.10	5.83	5.30	5.30	10.79
   len 2	5.98	4.05	3.41	2.93	10.78
   len 3	4.81	3.75	2.76	2.12	16.87
   len 4	4.58	4.25	2.93	2.25	13.90
   len 6	5.39	4.96	3.46	2.41	9.55
   len 8	6.24	5.47	3.61	2.36	7.51
   len 10	2.80	6.27	3.84	2.31	6.50
   len 12	2.93	6.88	4.02	2.29	5.35
   len 16	3.18	7.52	4.50	2.64	4.61
   len 20	3.36	7.94	4.76	2.90	3.90
   len 32	3.56	8.67	5.18	3.30	3.15
   len 45	3.67	8.87	5.42	3.52	2.63
   len 56	2.30	9.05	5.53	3.57	2.26
   len 64	2.35	9.12	5.57	3.60	2.19
   len 128	1.57	9.39	5.75	3.84	1.56
   len 256	1.36	9.55	5.75	3.89	1.34
   len 1024	0.71	9.60	5.92	4.01	0.70
   len 8192	0.38	9.65	5.91	4.00	0.38
  len 65536	0.28	9.69	5.95	4.04	0.28

nbits = 60
   len 1	7.00	5.70	5.23	5.27	10.56
   len 2	5.80	3.90	3.30	2.86	10.55
   len 3	4.67	3.65	2.69	2.08	16.51
   len 4	4.45	4.14	2.84	2.20	13.61
   len 6	5.31	4.92	3.42	2.35	9.97
   len 8	6.22	5.45	3.59	2.34	8.94
   len 10	2.74	6.31	3.82	2.27	7.67
   len 12	2.90	6.83	4.03	2.28	6.76
   len 16	3.19	7.54	4.44	2.63	5.66
   len 20	3.32	7.99	4.81	2.95	4.94
   len 32	3.60	8.68	5.17	3.31	4.28
   len 45	3.71	8.83	5.44	3.56	3.58
   len 56	3.13	8.98	5.49	3.60	3.13
   len 64	2.89	9.13	5.60	3.70	2.88
   len 128	2.22	9.30	5.73	3.87	2.21
   len 256	2.11	9.56	5.83	3.95	2.10
   len 1024	1.45	9.59	5.94	4.03	1.45
   len 8192	1.07	9.69	5.97	4.06	1.06
  len 65536	0.96	9.64	5.97	4.06	0.94

nbits = 62
   len 1	6.99	5.74	5.24	5.29	10.73
   len 2	5.88	3.99	3.34	2.86	10.59
   len 3	4.74	3.70	2.72	2.11	16.73
   len 4	4.50	4.17	2.85	2.21	13.66
   len 6	5.31	4.91	3.43	2.38	10.10
   len 8	6.25	5.30	3.52	2.34	9.07
   len 10	2.76	6.31	3.87	2.29	7.71
   len 12	2.89	6.67	3.97	2.29	6.82
   len 16	3.20	7.61	4.53	2.70	5.77
   len 20	3.33	7.82	4.74	2.95	4.99
   len 32	3.58	8.67	5.17	3.36	4.31
   len 45	3.69	8.68	5.43	3.57	3.66
   len 56	3.19	9.10	5.58	3.64	3.16
   len 64	2.90	9.12	5.64	3.72	2.95
   len 128	2.27	9.44	5.79	3.83	2.26
   len 256	2.14	9.56	5.84	3.96	2.17
   len 1024	1.61	9.66	5.92	4.01	1.60
   len 8192	1.22	9.68	5.99	4.07	1.23
  len 65536	1.06	9.65	5.99	4.08	1.05

nbits = 63
   len 1	7.10	5.78	5.31	5.28	10.78
   len 2	5.98	4.00	3.42	2.90	10.72
   len 3	4.77	3.72	2.75	2.13	17.00
   len 4	4.55	4.20	2.90	2.24	13.82
   len 6	5.31	4.98	3.46	2.40	10.19
   len 8	6.17	5.43	3.54	2.33	9.10
   len 10	3.79	6.33	3.86	2.27	7.79
   len 12	4.04	6.86	4.00	2.26	6.87
   len 16	4.31	7.54	4.52	2.66	5.85
   len 20	4.54	7.96	4.78	2.93	5.08
   len 32	4.81	8.73	5.21	3.37	4.63
   len 45	5.03	8.99	5.42	3.54	3.98
   len 56	3.56	9.17	5.60	3.69	3.52
   len 64	3.35	9.07	5.48	3.69	3.36
   len 128	2.75	9.46	5.82	3.86	2.71
   len 256	2.69	9.57	5.81	3.98	2.69
   len 1024	1.85	9.70	5.99	4.02	1.86
   len 8192	1.34	9.54	5.90	4.00	1.34
  len 65536	1.22	9.77	5.96	4.08	1.19

nbits = 64
   len 1	7.12	5.80	 na 	 na	10.86
   len 2	5.52	4.04	 na 	 na	10.71
   len 3	4.48	3.74	 na 	 na	14.14
   len 4	4.54	4.23	 na 	 na	12.87
   len 6	5.34	4.95	 na 	 na	9.88
   len 8	5.95	5.44	 na 	 na	9.00
   len 10	6.61	6.23	 na 	 na	7.99
   len 12	7.31	6.86	 na 	 na	6.97
   len 16	6.46	7.46	 na 	 na	6.31
   len 20	5.80	7.98	 na 	 na	5.57
   len 32	4.78	8.55	 na 	 na	4.74
   len 45	4.16	8.94	 na 	 na	4.12
   len 56	3.64	9.09	 na 	 na	3.61
   len 64	3.52	9.21	 na 	 na	3.46
   len 128	2.67	9.48	 na 	 na	2.65
   len 256	2.82	9.66	 na 	 na	2.78
   len 1024	1.99	9.64	 na 	 na	1.99
   len 8192	1.47	9.69	 na 	 na	1.44
  len 65536	1.29	9.76	 na 	 na	1.26

[vneiger]

Looks very nice!

This is might obsolete some of the optimizations proposed in #2492 (others may be complementary).

Special cases like evaluation at 1 or -1 still require special attention I guess, but apart from that, the solution in this PR with $\sqrt{n}$ splitting looks much better than the version with $4$-splitting via loop unrollings that was proposed in #2492. Maybe the latter should still be considered for small-ish lengths, I'll keep this in mind when revisiting #2492 .

[fredrik-johansson]

Some ideas to improve the rectangular algorithm, if you're interested:

• Improve instruction level parallelism generating the table of powers
• Hardcode the dot products for a few small lengths m and incorporate the addition without an nmod_add
• Use lazy reductions
• Optimize the splitting parameter empirically
• For short lengths, pick m and r without the square root and division