The optimum settings is to find a balance between large enough sample intervals to achieve high resolution per individual frequency sample, max. 10% of the Frequency deviation.
And small enough Sample intervals to capture enough frequency samples per modulation cycle for good graph visibility.
A rule of thumb is that the number of samples per modulation cycle should be >10, for good graphical view of the modulation signal, and acceptable error of fmax and fmin
Example: 10 kHz modulation frequency (100 us modulation cycle) of a 200 MHz carrier with 200 kHz deviation (0.1% modulation).
Set Sample interval to 10 μs (10% of the modulation cycle). Set Sample Count (N) to 100 (to cover 10 modulation cycles).
Now every frequency sample will have a resolution of (10 ps/10 μs) x 200 MHz = 200 Hz.
This resolution is 1000 times better than the frequency deviation.
Start measurement and view the Timeline graph, which will show10 modulation cycles, with 10 samples per modulation cycle.
To improve the graphical experience, lower Sample Interval to 1 μs (1% of the modulation cycle), and increase Sample Count to 1000.
Now each frequency sample has a resolution of (10ps/1μs) x 200 MHz = 2 kHz. Still with a lot of margin to deviation.
You may want to play around with the Sample Interval and Sample Count setting until you have found your optimum view of the FM signal (no of displayed mod. cycles).