A lot of the ASET calculations, in 7974 for example, have no safety factor built into them, though some do. To be conservative, the fire engineer using these formulae should use conservative inputs.
RSET can be based on some calculations but a significant element, as we all know, is pre-movement time which cannot be calculated. It is best if the pre-movement time is engineered out of the equation by the use of rigorous, reliable and robust fire safety management procedures such as training, well practised evacuation procedures and the like.
If suitably, but not overly, pessimistic figures are derived for both ASET and RSET then there is little need for a large safety margin. But you would need to be confident about your results.
If you were to use average inputs into the various calculations instead of pessimistic ones then you would get average outputs and you would not know the full range of possible results and so you would have no idea how big your safety margin should be and no one would know if your building was really safe until it had a significant fire. [In general, this is a poor way of testing your hypotheses].
The more sensitivity analysis that is satisfactorily undertaken ['what if?' analysis], the more likely the building is to be safe.
I would rather see realistically pessimistic inputs and a small but adequate safety margin than average inputs and an arbitrarily large safety margin.